CrackML by @ml.with.umang
Interview questions / GenAI & LLM
GenAI & LLM interview question

Design a RAG system end to end.

Design a RAG system end to end. Structure your response as you would in a top-tier ML/AI engineering interview.

hardsystem designEvidence 75/1001 source reportAmazon

The 60-second answer

Define the knowledge boundary, chunking strategy, metadata, embeddings, and retrieval/index design. Separate retrieval quality from generation quality: measure recall/precision or nDCG, then faithfulness, citation correctness, task quality, safety, latency, and cost.

Build the answer in this order

1
Define the mechanism

Define the knowledge boundary, chunking strategy, metadata, embeddings, and retrieval/index design.

2
Explain the architecture

Separate retrieval quality from generation quality: measure recall/precision or nDCG, then faithfulness, citation correctness, task quality, safety, latency, and cost.

3
Compare trade-offs

Add reranking, evidence-aware prompting, insufficient-evidence fallbacks, and trace retrieval/model behavior for debugging.

4
Close with serving + evaluation

Test prompt injection, stale data, permissions, and corpus/model/version changes before production rollout.

A useful interview mental model

This is the shape of a strong answer—not a script to memorize.

01Requirements
02Data
03Model / Retrieval
04Serving
05Monitor

Senior-level signal

  • Version corpus, embeddings, reranker, prompt, and model independently so regressions are attributable and reversible.
  • For enterprise RAG, enforce authorization before retrieval and evaluate leakage/exfiltration attacks as first-class failure modes.

What the interviewer is really testing

Understanding beyond prompting: architecture, retrieval, evaluation, inference, safety, latency, cost, and failure recovery.

Likely follow-up questions

What changes at 10× traffic or data volume?
Which failure mode would you monitor first in production?
How would you evaluate this offline and online before rollout?

Common weak-answer patterns

  • Jumping to a model before defining the product contract.
  • Listing components without bottlenecks, metrics, or failure handling.
  • Ignoring data quality, serving latency, monitoring, and iteration.