CrackML by @ml.with.umang
Interview questions / ML Fundamentals
ML Fundamentals interview question

Why can the same algorithm produce different success rates on the same dataset?

Why can the same algorithm produce different success rates on the same dataset?

mediumconceptEvidence 31/1002 source reportsGoogleUber

The 60-second answer

Match the split strategy to the data-generating process instead of defaulting to random folds. Use stratified folds for class balance, grouped folds for entity leakage, and time-based splits for temporal prediction.

Build the answer in this order

1
Give the core idea

Match the split strategy to the data-generating process instead of defaulting to random folds.

2
Explain how it works

Use stratified folds for class balance, grouped folds for entity leakage, and time-based splits for temporal prediction.

3
Compare alternatives

Keep preprocessing and feature selection inside each training fold.

4
State failure modes + validation

Report variance across folds, not only the mean score.

A useful interview mental model

This is the shape of a strong answer—not a script to memorize.

01Definition
02Mechanism
03Trade-offs
04Failure modes
05When to use

Senior-level signal

  • For repeated entities or recommender data, design splits that reflect the future serving scenario.
  • Use nested cross-validation when model selection and performance estimation must be separated.

What the interviewer is really testing

Mechanistic understanding, assumptions, trade-offs, and whether you can turn a definition into a model decision.

Likely follow-up questions

What assumption makes this approach work?
When would you choose the strongest alternative instead?
What production or data failure mode changes your answer?

Common weak-answer patterns

  • Reciting a definition without mechanism or assumptions.
  • Claiming one technique is always better without a data regime.
  • Stopping before failure modes, validation, or deployment implications.