CrackML by @ml.with.umang
Interview questions / ML System Design
ML System Design interview question

Design a Q&A Bot

Design a production question-answering assistant over a large internal knowledge base.

hardsystem designEvidence 41/1001 source reportGoogle

The 60-second answer

Define answerability, source-of-truth scope, freshness, permissions, latency, and what the bot must do when evidence is insufficient. Build ingestion/chunking plus hybrid retrieval, optional reranking, grounded prompt construction, generation, and citation/abstention logic.

Build the answer in this order

1
Frame the problem

Define answerability, source-of-truth scope, freshness, permissions, latency, and what the bot must do when evidence is insufficient.

2
Design the data path

Build ingestion/chunking plus hybrid retrieval, optional reranking, grounded prompt construction, generation, and citation/abstention logic.

3
Choose the modeling stack

Evaluate retrieval recall separately from answer correctness, faithfulness, citation accuracy, latency, and task completion.

4
Serve, evaluate, iterate

Monitor stale documents, permission leaks, hallucination/abstention rates, prompt injection, and user feedback with a reviewed evaluation set.

A useful interview mental model

This is the shape of a strong answer—not a script to memorize.

01Requirements
02Data
03Model / Retrieval
04Serving
05Monitor

Senior-level signal

  • Senior answers isolate retrieval, generation, and policy failures so each can be debugged independently.
  • Include ACL-aware retrieval, versioned corpora, safe fallback, and cost/latency routing between model tiers.

What the interviewer is really testing

Product framing, data design, modeling choices, serving constraints, reliability, evaluation, and explicit trade-offs.

Likely follow-up questions

What changes at 10× traffic or data volume?
Which failure mode would you monitor first in production?
How would you evaluate this offline and online before rollout?

Common weak-answer patterns

  • Jumping to a model before defining the product contract.
  • Listing components without bottlenecks, metrics, or failure handling.
  • Ignoring data quality, serving latency, monitoring, and iteration.