CrackML by @ml.with.umang
Interview questions / ML Coding & PyTorch
ML Coding & PyTorch interview question

Implement and Optimize a Custom Neural Network Layer

Implement a custom neural-network layer and explain how you would profile and optimize it.

hardml codingEvidence 34/1001 source reportTikTok

The 60-second answer

Define tensor shapes, parameter initialization, forward semantics, dtype/device behavior, and whether broadcasting is intentional. Prefer vectorized framework operations first; verify gradients with autograd/gradcheck before optimizing.

Build the answer in this order

1
State tensor contract

Define tensor shapes, parameter initialization, forward semantics, dtype/device behavior, and whether broadcasting is intentional.

2
Implement the mechanism

Prefer vectorized framework operations first; verify gradients with autograd/gradcheck before optimizing.

3
Check numerics + gradients

Profile to identify kernel launch, memory movement, or compute bottlenecks; then consider fusion, layout changes, custom kernels, or reduced precision.

4
Test shapes and edge cases

Benchmark representative batch/shape regimes and preserve numerical/error tolerances against a reference implementation.

A useful interview mental model

This is the shape of a strong answer—not a script to memorize.

01Shapes
02Forward pass
03Loss / grads
04Numerics
05Tests

Senior-level signal

  • Senior answers optimize only after establishing a correctness oracle and profile evidence.
  • Discuss backward-pass cost, memory saved tensors, compile/fusion behavior, and portability across accelerators.

What the interviewer is really testing

Tensor fluency, shape reasoning, numerics, gradients, batching, device awareness, and the ability to debug—not API memorization.

Likely follow-up questions

What are the tensor shapes at each step?
Where could numerical instability or silent broadcasting appear?
How would you verify gradients and batched behavior?

Common weak-answer patterns

  • Ignoring shape, dtype, device, masking, or broadcasting assumptions.
  • Using a framework call without explaining the underlying operation.
  • Skipping gradient, numerical-stability, and batching checks.