The 60-second answer
Quantization reduces numeric precision of weights and/or activations to lower memory and improve throughput. Compare post-training quantization with quantization-aware training based on tolerated quality loss and retraining budget.
Build the answer in this order
1
Define the mechanism
Quantization reduces numeric precision of weights and/or activations to lower memory and improve throughput.
2
Explain the architecture
Compare post-training quantization with quantization-aware training based on tolerated quality loss and retraining budget.
3
Compare trade-offs
Evaluate per-channel/per-tensor schemes and outlier-sensitive layers on representative workloads.
4
Close with serving + evaluation
Benchmark end-to-end latency and memory on the target hardware; theoretical bit-width savings do not guarantee speedups.
A useful interview mental model
This is the shape of a strong answer—not a script to memorize.
01Definition
02Mechanism
03Trade-offs
04Failure modes
05When to use
Senior-level signal
- KV-cache precision can matter as much as weight precision for long-context serving.
- Use layer-wise sensitivity analysis rather than applying one precision uniformly.
What the interviewer is really testing
Understanding beyond prompting: architecture, retrieval, evaluation, inference, safety, latency, cost, and failure recovery.
Likely follow-up questions
What assumption makes this approach work?
When would you choose the strongest alternative instead?
What production or data failure mode changes your answer?
Common weak-answer patterns
- Reciting a definition without mechanism or assumptions.
- Claiming one technique is always better without a data regime.
- Stopping before failure modes, validation, or deployment implications.