CrackML by @ml.with.umang
Interview questions / GenAI & LLM
GenAI & LLM interview question

Embeddings for Retrieval vs Token Embeddings

How do dense retrieval embeddings differ from the token embeddings inside a language model?

mediumconceptEvidence 41/1001 source reportLinkedIn

The 60-second answer

Token embeddings are learned lookup/representation vectors used internally as part of sequence modeling; retrieval embeddings are trained/pool representations to make semantic similarity meaningful under a chosen distance metric. Retrieval encoders need objectives with positives/negatives and often normalize vectors for cosine/dot-product ANN search.

Build the answer in this order

1
Define the mechanism

Token embeddings are learned lookup/representation vectors used internally as part of sequence modeling; retrieval embeddings are trained/pool representations to make semantic similarity meaningful under a chosen distance metric.

2
Explain the architecture

Retrieval encoders need objectives with positives/negatives and often normalize vectors for cosine/dot-product ANN search.

3
Compare trade-offs

Evaluate retrieval embeddings by recall@K/MRR and hard-negative behavior, not by inspecting token-vector neighborhoods alone.

4
Close with serving + evaluation

Consider chunk/document pooling, dimensionality, domain adaptation, index refresh, and embedding-version compatibility.

A useful interview mental model

This is the shape of a strong answer—not a script to memorize.

01Definition
02Mechanism
03Trade-offs
04Failure modes
05When to use

Senior-level signal

  • Senior answers discuss representation anisotropy, hard-negative mining, and index/model version migration.
  • Explain why generative next-token training does not automatically yield optimal sentence/document retrieval geometry.

What the interviewer is really testing

Understanding beyond prompting: architecture, retrieval, evaluation, inference, safety, latency, cost, and failure recovery.

Likely follow-up questions

What assumption makes this approach work?
When would you choose the strongest alternative instead?
What production or data failure mode changes your answer?

Common weak-answer patterns

  • Reciting a definition without mechanism or assumptions.
  • Claiming one technique is always better without a data regime.
  • Stopping before failure modes, validation, or deployment implications.