Incident Response For ML

Why this matters

Machine learning systems fail in ways classic software doesn’t: data drift, degraded model quality, silent bias, broken features, and misbehaving online learners. A clear incident response plan reduces customer impact, protects revenue, and shortens recovery time.

• Real tasks you will do: set up model SLOs, triage alerts, execute rollback, coordinate cross-team comms, run post-incident review, and add guardrails to prevent repeats.
• Common stakes: incorrect predictions driving losses, unsafe outputs, compliance risk, and reputation damage.

Concept explained simply

Incident response for ML is a repeatable playbook you run when model behavior or the ML platform significantly deviates from expected performance or safety.

Mental model

• Think “smoke alarms + fire drill” for models: monitors detect smoke, you triage the fire, contain spread, restore normal service, then fireproof the house.
• Core loop: Detect → Triage → Contain → Recover → Verify → Learn.

Incident types, severities, and SLOs

Common ML incident types

• Data/feature issues: missing features, wrong ranges, schema drift, staleness.
• Model quality: accuracy drop, fairness regressions, concept drift.
• Serving/platform: latency spikes, timeouts, resource exhaustion, autoscaling issues.
• Training/ETL: failed jobs, corrupted artifacts, bad hyperparams pushed to prod.
• Safety/abuse: adversarial inputs, prompt injection, toxic outputs, PII leakage.

Severity (suggested)

• SEV1: Broad customer/business impact; unsafe or legally risky outputs.
• SEV2: Noticeable impact on KPIs; partial degradation.
• SEV3: Limited scope or early detection; workarounds exist.

ML SLO examples

• Prediction SLOs: p95 latency ≤ target; availability ≥ target.
• Quality SLOs: monitor proxy metrics (AUC, win rate, calibration error, PSI for drift) within thresholds.
• Safety SLOs: toxic or unsafe response rate ≤ threshold.

Worked examples

First 90 minutes: ML incident playbook

Exercises

Practice these to build reflexes. Then take the quick test at the end. Everyone can take the test; only logged-in users get saved progress.

Exercise 1 (ex1) — Draft a 1-page ML incident runbook

• Timebox: 25 minutes.
• Use your current or an imaginary ML service (e.g., churn model, ranking, LLM assistant).
• Include: ownership, SLOs, severity map, detection triggers, first-hour checklist, rollback/fallback rules, verification, comms template.

Exercise 2 (ex2) — Triage a drift alert

Given: PSI(feature_price)=0.32 (thr 0.2), p95 latency stable, error rate normal, A/B shows -3% conversion in mobile users only, last deploy 2 hours ago (feature normalization change).

• Decide SEV level and immediate containment.
• List top 3 diagnostic checks.
• Pick a rollback or patch approach and define verification metric.

Common mistakes and self-check

• Chasing perfect root cause before stabilizing service. Fix: rollback first.
• No quality proxies in prod. Fix: add drift, calibration, and safety monitors.
• Confusing infrastructure latency with model quality issues. Fix: separate dashboards and alerts.
• Over-broad changes during incidents. Fix: one change at a time; document.
• Poor comms cadence. Fix: schedule updates (e.g., every 30 minutes) with impact and ETA.

Practical projects

• Build a drift-and-safety dashboard with synthetic canaries; set actionable thresholds.
• Create a full runbook for one production model, including comms templates and decision tables.
• Run a 60-minute tabletop: simulate drift, execute rollback, post-incident review with action items.
• Implement a fallback path: last-known-good model + minimal feature set + rules-only mode.

Who this is for, prerequisites, and learning path

Who this is for

• Machine Learning Engineers, Data Scientists on-call, MLOps/Platform Engineers, and Product Engineers who own ML features.

Prerequisites

• Basic model evaluation (AUC, accuracy, calibration).
• Understanding of your serving stack and deployment process.
• Access to monitoring dashboards and logs (or mock equivalents).

Learning path

• Start with SLOs and alert thresholds.
• Define rollback/fallback mechanisms.
• Write a runbook; run a tabletop drill.
• Automate: canaries, circuit breakers, safety filters.
• Institutionalize: post-incident reviews and recurring checks.

Mini challenge

You receive: AUC -7% on desktop traffic; drift PSI high on feature_city; p95 latency OK; last content campaign launched 3 hours ago. In 8 bullet points, outline your first 30 minutes: assign roles, set SEV, immediate containment, top diagnostics, rollback plan, comms note, verification metrics, and next checkpoint.

Next steps

• Finish the exercises and ensure your 1-page runbook is ready.
• Run a 30-minute tabletop with a teammate.
• Take the quick test below to lock in concepts. Everyone can take it; only logged-in users get saved progress.

Quick Test

Answer a few questions to check your readiness. Passing score: 70%.