The 60-second answer
During training, dropout randomly zeros activations with probability p to reduce co-adaptation and act as regularization. With inverted dropout, surviving activations are scaled during training so no stochastic dropping or extra scaling is needed at inference.
Build the answer in this order
During training, dropout randomly zeros activations with probability p to reduce co-adaptation and act as regularization.
With inverted dropout, surviving activations are scaled during training so no stochastic dropping or extra scaling is needed at inference.
It is most useful when overfitting is a concern; too much dropout can increase bias and slow optimization.
Always switch the model to evaluation mode for deterministic inference.
A useful interview mental model
This is the shape of a strong answer—not a script to memorize.
Senior-level signal
- Discuss interactions with BatchNorm/LayerNorm and why dropout is used less aggressively in some modern Transformer blocks.
- Treat inference-mode mistakes as a correctness bug that can silently change output distributions.
What the interviewer is really testing
Likely follow-up questions
Common weak-answer patterns
- Reciting a definition without mechanism or assumptions.
- Claiming one technique is always better without a data regime.
- Stopping before failure modes, validation, or deployment implications.