Attention interview questions.
A curated set of 19 questions that repeatedly exercise Attention concepts in AI/ML interviews.
Explain Transformer architecture and each major component.GenAI & LLM · medium · Evidence 88/100Explain the attention mechanism in a Transformer.GenAI & LLM · medium · Evidence 78/100Implement self-attention.ML Coding & PyTorch · hard · Evidence 77/100Implement masked self-attention.ML Coding & PyTorch · hard · Evidence 77/100Implement single-head attention in PyTorch.ML Coding & PyTorch · hard · Evidence 75/100Explain attention in modern neural networks.GenAI & LLM · medium · Evidence 75/100Add a causal mask that hides future tokens.ML Coding & PyTorch · hard · Evidence 75/100What are sparse-attention methods in Transformers?GenAI & LLM · hard · Evidence 75/100Implement self-attention from scratch in code.ML Coding & PyTorch · hard · Evidence 75/100Implement multi-head attention in PyTorch.ML Coding & PyTorch · hard · Evidence 75/100Explain the architecture of attention.GenAI & LLM · medium · Evidence 75/100Explain embeddings, tokenization, attention, and BERTGenAI & LLM · medium · Evidence 41/100Explain the relationship between self-attention and Transformer representations.GenAI & LLM · medium · Evidence 41/100How do continuous batching and paged attention improve LLM serving?GenAI & LLM · hard · Evidence 41/100How does attention complexity affect long-document modeling?GenAI & LLM · hard · Evidence 41/100Attention Score From Q, K, VML Math · medium · Evidence 34/100Implement a Transformer BlockML Coding & PyTorch · hard · Evidence 34/100QKV in Multi-Head AttentionGenAI & LLM · medium · Evidence 34/100Code Multi-Head Self-AttentionML Coding & PyTorch · hard · Evidence 34/100