Model Registry And Artifact Management

Why this skill matters for MLOps Engineers

4) Organize artifact storage with retention and lifecycle policies

# Suggested object storage layout
models/
  iris_rf/
    1/  # model version
      model/...
      eval_report.json
      lineage.json
    2/
      ...
  churn_xgb/
    1/
      ...

# Example: lifecycle policy (conceptual)
policy:
  - rule: delete-nonprod-older-than-90d
    match: "models/*/*"
    if:
      stage: ["None", "Staging"]
      age_days: >= 90
    action: delete
  - rule: keep-last-5-prod
    match: "models/*/*"
    if:
      stage: ["Production"]
    action: keep_last: 5
  - rule: archive-eval-reports
    match: "models/*/*/eval_report.json"
    action: move_to: "archive/eval_reports/"

Retention reduces storage costs while preserving important Production history.

5) Build an audit trail for governance

{
  "timestamp": "2026-01-04T10:12:00Z",
  "actor": "mlops.engineer@company",
  "action": "PROMOTE",
  "model": "iris_rf",
  "from_stage": "Staging",
  "to_stage": "Production",
  "version": 3,
  "justification": "Passed QA tests, sign-off by risk officer",
  "run_id": "<run-id>",
  "git_commit": "<commit-sha>",
  "data_hash": "<sha256>"
}

Store audit entries immutably (append-only), reference them in model version tags, and include them in release notes.

Skill drills (do-now checklist)

• Create a naming convention: model_name, owner, business domain, risk level.
• Log at least 5 metadata fields per run: dataset version, git commit, env, training date, evaluator.
• Define numeric thresholds for promotion gates (e.g., F1 ≥ 0.92, latency ≤ 30ms).
• Write a retention policy for Non-Production versions older than 90 days.
• Produce a sample audit record for a rollback and store it immutably.

Common mistakes and debugging tips

• Mistake: Storing massive datasets inside the registry. Tip: Keep big data in object storage; log a pointer and hash in metadata.
• Mistake: Missing environment details. Tip: Log requirements.txt/conda.yaml and the runtime image tag.
• Mistake: Ambiguous model names. Tip: Enforce naming + owner tags; reject uploads without required tags.
• Mistake: One-step to Production. Tip: Use Staging with canary/AB checks before full rollout.
• Mistake: No rollback plan. Tip: Always keep last N Production versions; document rollback steps.
• Debug tip: If a model is not reproducible, compare git_commit, data_hash, and env specs first—most issues stem from mismatches there.

Mini project: Productionize a classifier with governance

1. Train a small model and register it with parameters, metrics, lineage.json, and an evaluation report artifact.
2. Implement a promotion script: Dev → Staging when metrics pass; require an approved_by tag for Prod.
3. Set up a storage layout and a simple retention policy keeping the last 5 Production versions.
4. Create an audit entry for both promotion and a simulated rollback, store them immutably, and tag the model with their IDs.
5. Demonstrate reproducibility by re-running training from git_commit and data_hash to recreate the same metrics within tolerance.

Next steps

• Automate promotion and rollback in CI/CD.
• Add drift monitoring outputs as artifacts and block promotions on drift alerts.
• Introduce model cards and risk scores; require sign-off for high-risk categories.

Subskills

• Model Versioning And Metadata — Log parameters, metrics, tags, and artifacts; adopt semantic versioning; ensure reproducibility. Est. 60–90 min.
• Stage Promotion Dev Stage Prod — Implement Dev → Staging → Prod with automated checks and human approvals. Est. 45–75 min.
• Approval And Governance Flows — Add sign-offs, risk labels, model cards, and audit evidence. Est. 45–90 min.
• Linking Model To Data And Code — Attach git commit, data hashes, feature schema, and environment specs. Est. 45–90 min.
• Artifact Storage And Retention — Organize object storage, encryption, lifecycle, and size limits. Est. 45–75 min.
• Audit Trails And Traceability — Record who changed what, when, and why; maintain lineage for reproducibility. Est. 45–75 min.

Quick FAQ