Model Versioning And Canary Releases

Who this is for

Computer Vision Engineers and ML practitioners who deploy models to production and need safe, observable rollouts without disrupting users or downstream systems.

Prerequisites

• [ ] Basic understanding of model serving (REST/gRPC endpoints)
• [ ] Comfort with containers and environment configuration
• [ ] Familiarity with metrics (latency, error rate) and model quality (precision/recall)

Learning path

1. Learn version identifiers: semantic, timestamp, and data-aware versions.
2. Set up a registry or naming convention for models and Docker images.
3. Add observability: request IDs, model version tags, and key metrics.
4. Practice canary strategies: shadow traffic, weighted routing, and rollback.
5. Automate checks and rollout gates.

Why this matters

In real computer vision systems (content moderation, defect detection, OCR, retail analytics), a model update can improve accuracy but also risk outages or quality regressions. Versioning makes changes traceable and reversible. Canary releases let you test new models with a small slice of traffic, measure impact, and roll forward or back safely.

Concept explained simply

Model versioning is giving every model build a unique label and storing its exact recipe (code, data, parameters, and artifacts). Canary release is rolling out a new model to a small percentage of users first, checking health and quality, and only then increasing the share.

Mental model

Think of a dimmer switch, not an on/off switch. You gradually turn up the brightness (traffic to the new model) while watching gauges (metrics). If any gauge goes red, you turn the brightness down instantly (rollback).

Versioning strategies

At minimum, label: model version, code commit, dataset tag, training config hash, and serving image tag.

Canary release patterns

• Shadow (mirror) testing: New model receives a copy of requests but does not affect user responses. Compares outputs offline.
• Weighted routing: X% of live traffic hits the new model; users see its results. Increase weight if metrics are good.
• Sticky cohorts: Keep specific users/devices on one version to avoid flip-flop outputs affecting UX.
• Feature gating: Use flags to enable/disable new model features without redeploy.

Metrics and guardrails

• System: p50/p95 latency, error rate, timeouts, GPU/CPU utilization, memory.
• Model: precision/recall/F1, IoU (segmentation), mAP (detection), OCR CER/WER.
• Business: false reject/accept rates, throughput, cost per 1k inferences.

Set guardrails, e.g.: "p95 latency ≤ baseline + 10%", "mAP drop < 1 point", "error rate < 0.5%". Automate alerts and stop-the-world rollback if violated.

Worked examples

Step-by-step: Implement a canary for a CV inference service

1. Tag everything: Model: cv-detector:2.0.0+data=b4f7a1; Image: cv-detector:serve-2026-01-05.
2. Expose metrics: Add model_version to logs/metrics. Include request_id and cohort_id.
3. Shadow stage: Mirror 100% of traffic to v2; store predictions and latency. Compare to v1 on a labeled slice.
4. Define guardrails: e.g., mAP ≥ v1−1pt; p95 latency ≤ v1+10%.
5. Weighted rollout: 5% → 25% → 50% → 100%, each for 30–120 min with automated checks.
6. Rollback plan: One-click route-to-v1; keep v2 pods warm for quick retry later.
7. Post-release: Freeze v1 as stable; archive artifacts and training manifest.

Readiness checklist

• [ ] Version tags for model, code, data, and image are recorded
• [ ] Telemetry includes model_version and request_id
• [ ] Shadow results reviewed and signed off
• [ ] Guardrails and rollback triggers documented
• [ ] Rollout stages and owners assigned
• [ ] Post-release archiving plan in place

Exercises

Do the tasks below. The quick test is available to everyone; sign in to save your progress.

# Pseudo YAML for weighted routing (conceptual)
service: seg-service
versions:
  - name: v1-4-3
    weight: 95
  - name: v1-5-0
    weight: 5
metrics_guardrails:
  p95_latency_increase_pct: <= 10
  error_rate_pct: <= 0.5
  iou_drop_points: <= 1
rollout_stages:
  - {weight: 5,  duration: 60m}
  - {weight: 25, duration: 120m}
  - {weight: 50, duration: 180m}

# Conceptual routing change per stage (weights change only)
versions:
  - {name: v1-4-3, weight: 95}
  - {name: v1-5-0, weight: 5}
# ... then 75/25, then 50/50

Common mistakes and self-check

• No data version recorded: Self-check: Can you reproduce training with the exact dataset tag/hash?
• Only system metrics monitored: Self-check: Do you track class-wise metrics and business KPIs?
• Skipping shadow tests: Self-check: Do you have evidence the new model performs on key cohorts before user-facing traffic?
• Unclear rollback: Self-check: Can you revert traffic in one action and within one minute?
• Non-sticky canary users: Self-check: Are canary users consistently routed to the same version?

Practical projects

1. Build a mini model registry: Store model artifact, dataset tag, and metrics in a simple manifest (JSON) and serve it via your API.
2. Shadow inference pipeline: Duplicate incoming requests to a second model and compute delta metrics offline.
3. Guardrail dashboard: Visualize p50/p95 latency, error rate, and two model metrics with per-version filters.

Mini challenge

Write a one-page runbook for canarying a new defect detection model in a GPU-constrained environment. Include pre-warm strategy, rollout stages, guardrails, and rollback steps. Aim for clear, actionable bullet points.

Next steps

• Automate rollout gates with metric thresholds.
• Add cohort-level evaluation reports to every release.
• Standardize naming: model, data, and image tags in every log line.