CrackML by @ml.with.umang
Interview questions / GenAI & LLM
GenAI & LLM interview question

Why use RAG instead of fine-tuning an LLM?

Why use RAG instead of fine-tuning an LLM?. Structure your response as you would in a top-tier ML/AI engineering interview.

mediumconceptEvidence 66/1002 source reportsAmazonGoogle

The 60-second answer

Define the knowledge boundary and chunk/index approved documents with metadata needed for filtering and access control. At query time, retrieve candidates, optionally rerank them, then construct a prompt that clearly separates instructions from retrieved evidence.

Build the answer in this order

1
Define the mechanism

Define the knowledge boundary and chunk/index approved documents with metadata needed for filtering and access control.

2
Explain the architecture

At query time, retrieve candidates, optionally rerank them, then construct a prompt that clearly separates instructions from retrieved evidence.

3
Compare trade-offs

Evaluate retrieval recall/precision separately from answer faithfulness, citation correctness, task quality, latency, and cost.

4
Close with serving + evaluation

Add fallbacks for insufficient evidence and log retrieval/model traces for debugging.

A useful interview mental model

This is the shape of a strong answer—not a script to memorize.

01Definition
02Mechanism
03Trade-offs
04Failure modes
05When to use

Senior-level signal

  • Separate retrieval failures from generation failures and version the corpus, embeddings, prompts, and model independently.
  • For enterprise use, enforce permissions before retrieval and test prompt injection/data-exfiltration attacks.

What the interviewer is really testing

Understanding beyond prompting: architecture, retrieval, evaluation, inference, safety, latency, cost, and failure recovery.

Likely follow-up questions

What assumption makes this approach work?
When would you choose the strongest alternative instead?
What production or data failure mode changes your answer?

Common weak-answer patterns

  • Reciting a definition without mechanism or assumptions.
  • Claiming one technique is always better without a data regime.
  • Stopping before failure modes, validation, or deployment implications.