CrackML by @ml.with.umang
Interview questions / ML Fundamentals
ML Fundamentals interview question

Why can the same algorithm produce different success rates on the same dataset?

mediumconceptEvidence 31/1002 source reportsGoogleUber

The 60-second answer

Match the split strategy to the data-generating process instead of defaulting to random folds. Use stratified folds for class balance, grouped folds for entity leakage, and time-based splits for temporal prediction.

Build the answer in this order

1
Give the core idea

Match the split strategy to the data-generating process instead of defaulting to random folds.

2
Explain how it works

Use stratified folds for class balance, grouped folds for entity leakage, and time-based splits for temporal prediction.

3
Compare alternatives

Keep preprocessing and feature selection inside each training fold.

4
State failure modes + validation

Report variance across folds, not only the mean score.

A useful interview mental model

This is the shape of a strong answer—not a script to memorize.

01Definition
02Mechanism
03Trade-offs
04Failure modes
05When to use

Senior-level signal

  • For repeated entities or recommender data, design splits that reflect the future serving scenario.
  • Use nested cross-validation when model selection and performance estimation must be separated.

What the interviewer is really testing

Mechanistic understanding, assumptions, trade-offs, and whether you can turn a definition into a model decision.

Likely follow-up questions

What assumption makes this approach work?
When would you choose the strongest alternative instead?
What production or data failure mode changes your answer?

Common weak-answer patterns

  • Reciting a definition without mechanism or assumptions.
  • Claiming one technique is always better without a data regime.
  • Stopping before failure modes, validation, or deployment implications.