SFT vs DPO vs RLHF vs GRPO Interview Guide
Compare modern LLM post-training methods by supervision signal, optimization loop, data requirements, stability, and evaluation.
Start with SFT
Supervised fine-tuning learns from target demonstrations and is usually the first post-training stage for task behavior, formatting, or instruction following. Its ceiling depends heavily on demonstration quality and coverage.
Preference optimization
DPO trains directly from chosen-vs-rejected preference pairs without a separate online RL loop. Contrast that with classical RLHF pipelines that fit a reward model and optimize a policy against that learned reward.
GRPO and online reward optimization
GRPO is an online reinforcement-learning-style method that scores sampled outputs with reward functions and updates the policy relative to groups of generations. Discuss reward design, sampling cost, variance, reward hacking, and stability rather than reducing it to an acronym.
How to choose
Choose based on the missing capability and available signal: demonstrations for SFT, preference pairs for DPO, and reliable verifiable or learned rewards for online methods. Always hold out behavior, safety, and capability regression sets.