The 60-second answer
Project token representations into queries, keys, and values, then compute scaled dot-product scores QKᵀ/√d. Apply masking when needed, softmax the scores, and use them to take a weighted sum of V.
Build the answer in this order
1
Define the mechanism
Project token representations into queries, keys, and values, then compute scaled dot-product scores QKᵀ/√d.
2
Explain the architecture
Apply masking when needed, softmax the scores, and use them to take a weighted sum of V.
3
Compare trade-offs
Multi-head attention performs this in parallel subspaces so different heads can model different relationships.
4
Close with serving + evaluation
Track tensor shapes, masking semantics, numerical stability, and O(n²) sequence-length cost.
A useful interview mental model
This is the shape of a strong answer—not a script to memorize.
01Definition
02Mechanism
03Trade-offs
04Failure modes
05When to use
Senior-level signal
- Discuss memory bandwidth and attention/KV-cache costs, not only FLOPs, when reasoning about real latency.
- Tie architectural choices to the task, context length, training objective, and deployment constraints.
What the interviewer is really testing
Understanding beyond prompting: architecture, retrieval, evaluation, inference, safety, latency, cost, and failure recovery.
Likely follow-up questions
What assumption makes this approach work?
When would you choose the strongest alternative instead?
What production or data failure mode changes your answer?
Common weak-answer patterns
- Reciting a definition without mechanism or assumptions.
- Claiming one technique is always better without a data regime.
- Stopping before failure modes, validation, or deployment implications.