LLM Guardrails & Model Routing Interview Guide
Design layered safety guardrails and model-routing systems that balance quality, latency, cost, policy risk, and fallback behavior.
Layered guardrails
Separate input validation, policy classification, retrieval/tool authorization, model constraints, output checks, and human review. Prompt instructions alone are not an authorization boundary.
Routing policy
Route by task complexity, modality, context length, latency SLO, risk level, customer tier, and observed model quality. Use deterministic rules where possible and learned routers only when their failure modes are measurable.
Fallbacks
Define what happens on refusal, timeout, low confidence, tool failure, safety trigger, or capacity pressure: retry, smaller/larger model, alternate provider, deterministic workflow, or human escalation.
Evaluation and monitoring
Track route distribution, quality by route, fallback frequency, safety violations, latency, cost, and drift. Shadow-test routing changes and maintain hard policy regression suites.