Feature Lineage And Governance

Why this matters

Core elements you should capture

• Ownership: feature owner, reviewer, on-call rotation.
• Provenance: source systems, tables, event topics, external files.
• Transformations: code location or job name, dependency versions, data contracts, validation checks.
• Time dimensions: as-of timestamp, windowing, backfill ranges, training cutoffs.
• Versions: semantic version for feature definition (e.g., v1.2.0), data schema hash, model compatibility.
• Stores: offline dataset path(s) and partitioning; online store keys, TTL, freshness SLA.
• Quality signals: null rate, freshness lag, drift metrics, test status.
• Access & PII: sensitivity label, approved roles, masking policy, audit log references.
• Lifecycle: status (active, deprecated, archived), replacement feature, deprecation date and plan.

Worked examples

Example 1: Cancellation risk feature impact analysis

Feature: user_booking_cancel_rate_30d:v2

• Sources: bookings.events (stream), refunds.table (batch)
• Transform: sliding 30-day window aggregation job (Airflow: booking_features_30d)
• Offline: /feature_store/booking/30d_cancel_rate/v2/partition=2025-05-01
• Online: Redis key=user_id, TTL=36h
• PII: none
• Models consuming: cancellation_risk_model:v4

Change request: refunds.table schema adds refund_reason. Using lineage, you see the feature does not read refund_reason; impact is low. You still update tests to ensure no breakage. Governance record: change reviewed, tests passed, approved by owner.

Example 2: PII reduction and deprecation

Feature: user_last4_card_digits:v1 (PII: limited)

Policy requires minimizing PII in features. Replacement: user_has_saved_card:v1 (boolean). Governance steps:

• Mark old feature status=deprecated; set sunset date in 60 days.
• Create migration note and notify model owners.
• Add access policy for old feature: read only for whitelisted services during migration.
• Audit: confirm all consumers switched before sunset; archive lineage record.

Example 3: Regulator asks to reproduce a decision

Decision date: 2025-11-03 14:22 UTC. Model: credit_approval_model:v7. Steps:

1. Find prediction log with model version and feature vector hash.
2. Use lineage to retrieve offline snapshot as-of 2025-11-03 14:20 UTC.
3. Confirm feature definitions (v7-compatible) and dependency versions.
4. Recompute features with time travel or load stored snapshot; verify equality to logged vector.
5. Export report: data sources, code commit IDs, validation checks, approvals.

Outcome: fully reproducible evidence trail.

Who this is for

• Machine Learning Engineers integrating a feature store with batch/stream pipelines.
• Data Scientists who publish features for reuse across models.
• Platform/ML Ops engineers standardizing metadata and compliance.

Prerequisites

• Comfort with basic data modeling and ETL/ELT concepts.
• Understanding of offline vs. online feature stores and training/serving skew.
• Familiarity with CI/CD and code reviews.

Learning path

1. Map a single feature end-to-end (source → transform → store → model).
2. Add governance metadata: owner, PII label, SLA, version, lifecycle.
3. Automate lineage capture (from jobs) and validation checks.
4. Run an impact analysis drill and a deprecation drill.
5. Practice reproduction of a past prediction using as-of time travel.

How to implement lineage and governance (practical steps)

1. Define a feature contract: name, keys, schema, null policy, time semantics, PII label, owner, reviewers.
2. Version features: semantic version on definition changes; immutable offline snapshots; strict compatibility notes.
3. Annotate pipelines: include job name, code commit, dependency versions, input datasets, schedule, and validation results.
4. Capture time travel: record as-of timestamp, windowing, backfill start/end, and training cutoff logic.
5. Set access controls: approved roles, masking, TTL, de-identification notes.
6. Automate checks: schema drift, null thresholds, freshness SLAs, and drift metrics with alerting.
7. Lifecycle management: statuses (draft → active → deprecated → archived), with deprecation plans and replacement pointers.
8. Audit and approvals: require reviewer sign-off for PII features and breaking changes; store logs.

Common mistakes and how to self-check

• Missing time context: You cannot reproduce decisions. Self-check: can you fetch the exact snapshot as-of a timestamp?
• Unversioned definitions: Consumers silently break. Self-check: does every breaking change bump the major version?
• Ignoring online/offline parity: Skew in production. Self-check: are transformations shared or validated for parity?
• PII leakage: Overbroad access. Self-check: does each feature have a sensitivity label and masking policy?
• Orphaned features: No owner to fix issues. Self-check: is an on-call owner listed?
• Weak deprecation discipline: Legacy debt accumulates. Self-check: do you set sunset dates and monitor consumer migration?

Exercises

Do these to lock in the concepts. They mirror the interactive exercises below.

1. ex1 — Create a minimal lineage record: Pick a real or fictional feature and write a concise lineage + governance record including owner, sources, transforms, versions, stores, PII, and lifecycle.
2. ex2 — Plan a safe deprecation: Choose a feature to replace, define status changes, migration plan, and acceptance criteria.

• Checklist:
  • Owner and reviewers are named.
  • Source tables/topics and transforms are listed.
  • As-of time and versioning are clear.
  • PII label and access policy set.
  • Lifecycle status and next action defined.

Practical projects

• Build a feature lineage template and populate it for three features (batch, streaming, and hybrid).
• Implement CI checks that fail PRs when a feature changes schema without a version bump.
• Create a deprecation playbook and run a mock deprecation with stakeholders.
• Produce a reproduction report for a past prediction using as-of snapshots.

Quick Test

Short, interactive quiz. Available to everyone; only logged-in users get saved progress.

Mini challenge

Pick one of your existing features. In 30 minutes, write a one-page lineage + governance record. Then ask a teammate to find one missing detail. Update your record, define a deprecation or improvement action, and schedule it.

Next steps

• Apply the template to your top 5 features.
• Automate metadata collection from your pipelines.
• Practice an impact analysis drill monthly.
• Keep a simple changelog in each feature definition.