Logging Inputs Outputs And Metadata

Why this matters

In production, your model will make thousands or millions of decisions. When performance dips, users complain, or auditors ask “why?”, good logs are your only reliable record. Logging inputs, outputs, and metadata lets you: diagnose data drift and outages, reproduce decisions for audits, track model versions across A/B tests, measure latency and reliability against SLOs, and close the loop when labels arrive later.

Concept explained simply

Think of logging as a flight recorder for your model. Each prediction creates a compact record of what came in (inputs), what went out (outputs), and the context (metadata). With this record, you can replay, explain, and improve your system.

Mental model

• Inputs: the facts the model saw (safely captured).
• Outputs: the decision and confidence scores.
• Metadata: everything needed to interpret the above (model + data versions, time, environment, latency, request/trace IDs).

What to log

Minimal logging set (checklist)

• request_id (unique, UUID or ULID)
• timestamp (UTC, ISO 8601)
• model_version and schema_version
• input summary (hashes, shapes, feature names, safe aggregates)
• prediction output (class/regression value)
• score or probability and decision_threshold (if applicable)
• latency_ms and status (ok/error code)
• environment (prod/staging) and endpoint name

Enhanced observability (nice-to-have)

• data_version or feature_store_commit
• feature schema hash (to detect silent changes)
• top_k results and scores (for ranking/recs)
• explanations summary (e.g., top 3 feature attributions)
• trace_id/span_id to connect app, model, and DB logs
• feedback label and label_timestamp (when known later)

Worked examples

Example 1 — Fraud classifier (binary)

{
  "request_id": "01JABC...",
  "timestamp": "2026-01-01T12:00:00Z",
  "model_version": "fraud_v17",
  "schema_version": "1.2",
  "env": "prod",
  "input": {"amount_bucket": "100-200", "merchant_hash": "h:9fa2", "device_fingerprint": "h:0c11"},
  "prediction": 1,
  "score": 0.83,
  "decision_threshold": 0.7,
  "latency_ms": 42,
  "status": "ok"
}

How it helps: investigate spikes by filtering merchant_hash and amount_bucket; compare score distributions across model versions.

Example 2 — Toxicity moderation (NLP)

{
  "request_id": "01JXYZ...",
  "timestamp": "2026-01-01T12:01:00Z",
  "model_version": "tox_v5",
  "schema_version": "2.0",
  "env": "prod",
  "input_summary": {"text_len": 137, "lang": "en"},
  "prediction": "toxic",
  "scores": {"toxic": 0.91, "safe": 0.09},
  "latency_ms": 18,
  "status": "ok"
}

Privacy: we log text_len and lang, not full text. Full payloads may be sampled to a secured store if policy allows.

Example 3 — Recommendations (top-k ranking)

{
  "request_id": "01JREC...",
  "timestamp": "2026-01-01T12:02:00Z",
  "model_version": "recs_v3",
  "schema_version": "1.0",
  "env": "prod",
  "user_surrogate": "u:h_2bf0",
  "candidates": 320,
  "top_k": [
    {"item": "i:918", "score": 0.82},
    {"item": "i:144", "score": 0.79},
    {"item": "i:553", "score": 0.77}
  ],
  "latency_ms": 63,
  "status": "ok"
}

Later, when clicks/conversions arrive, append feedback: clicked_item, label_timestamp.

Design patterns that help

• Schema versioning: add schema_version; evolve via additive changes; keep migration notes.
• Correlation: request_id + trace_id tie app, feature store, and model logs.
• Late labels: log inference now; append feedback later with same request_id.
• Idempotency: if you retry logging, deduplicate by request_id.
• Batch vs. online: for batch scoring, include batch_id and record_index.

Who this is for

• Machine Learning Engineers deploying and maintaining models.
• Data/ML Ops teams building monitoring pipelines.
• Developers integrating model endpoints into applications.

Prerequisites

• Basic Python or similar for reading/writing JSON records.
• Familiarity with your model’s input/output shapes.
• Understanding of privacy basics (PII, hashing, redaction).

Learning path

1. Define a minimal schema and implement request_id + schema_version.
2. Add model_version, data_version, and latency logging.
3. Introduce privacy controls (hashing, truncation, sampling).
4. Wire correlation with trace_id and endpoint logs.
5. Append feedback labels and compute monitoring metrics.

Exercises

These match the graded tasks below. Do them here, then check your answers.

Common mistakes and self-check

• Logging raw PII (fix: hash/truncate or store only summaries).
• Omitting model_version (fix: stamp every record, including A/B buckets).
• Not logging threshold (fix: store the decision rule used at inference time).
• Ignoring errors/timeouts (fix: log status and reason; alert on spikes).
• Unbounded payload size (fix: cap field sizes and top_k length).
• No correlation IDs (fix: add request_id/trace_id and propagate).

Self-check: pick any record. Can you tell exactly which model, with which inputs and rule, produced the output at a specific time, and how long it took? If not, your schema is missing pieces.

Practical projects

• Add logging to a REST prediction service and build a daily report of latency and score distributions.
• Implement adaptive sampling that always keeps errors and 5% of normal traffic.
• Create a label-joining job that attaches outcomes to inference logs using request_id.

Next steps

• Wire aggregate dashboards: latency percentiles, error rate, data/score drift.
• Define alerts on missing fields, version mismatches, and drift thresholds.
• Automate schema validation in CI to prevent breaking changes.

Mini challenge

In 10 minutes, list the exact fields you need to explain one surprising prediction your stakeholders asked about last week. Compare your list to the Minimal logging set and fill any gaps.

Quick test

Ready to check your understanding? Take the quick test below. The quick test is available to everyone; only logged-in users get saved progress.