Data Quality SLAs And Ownership

Why this matters

As an Analytics Engineer, your work powers dashboards, decisions, and downstream automations. Clear Data Quality SLAs (service level agreements) and ownership make quality predictable. Without them, you face silent data drift, unclear escalation, and blame loops. With them, the right people respond quickly, stakeholders trust data, and you protect business outcomes.

• Real tasks you will do: define freshness targets for critical models, agree response times for incidents, set ownership boundaries across producer/transform/consumer teams, and document runbooks and exception windows.
• Impact: fewer broken dashboards, faster fixes, and a shared language for quality trade-offs.

Concept explained simply

Think of data products like public transport. Riders expect buses to arrive on time (freshness), have room available (completeness), and go to the right stops (accuracy/consistency). An SLA is the timetable plus escalation if a bus is late.

• SLI (Indicator): how you measure. Example: minutes from source extract to model availability.
• SLO (Objective): the target. Example: 95% of days the model is ready by 06:30 UTC.
• SLA (Agreement): the commitment and what happens if you miss. Example: on-call within 15 minutes, stakeholder notice, root cause analysis within 24 hours.

Key components of a Data Quality SLA

• Scope: which datasets/models, which business use cases (e.g., Finance close, Daily Ops).
• SLIs: clear, machine-measurable metrics (e.g., max lag in minutes; % non-null keys; reconciliation % vs source).
• SLOs: target levels and measurement window (e.g., 99% of runs per calendar month).
• Error budget: how many allowed misses before corrective actions.
• Incident severity and response: severity levels, response and resolution times.
• Ownership (RACI): Responsible, Accountable, Consulted, Informed per dataset and pipeline.
• Monitoring and evidence: where metrics live and how they alert.
• Exceptions and change management: temporary waivers with end dates; process for updating the SLA.
• Review cadence: quarterly review with producers and consumers.

Ownership model

Ownership clarifies who fixes what. Use a simple RACI per dataset and per pipeline step.

• Producers (source systems): own upstream correctness and schema changes.
• Analytics Engineering: owns transformations, tests, and data models exposed to consumers.
• Data Stewards/Business Owners: define business rules, criticality, and accept SLOs.
• BI/Consumers: report issues, validate business fit, and help prioritize.

Worked examples

Example 1: Daily finance model freshness

• SLI: minutes from last successful extract to model availability.
• SLO: 99% of business days model ready by 06:30 UTC.
• Error budget: up to 2 misses/month.
• Severity: Sev-2 if 06:30–07:00; Sev-1 after 07:00 on business days.
• Ownership: AE on-call Responsible; Finance steward Consulted.
• Escalation: page on-call; status note to Finance within 15 minutes; RCA in 24 hours.

Example 2: Event stream completeness

• SLI: % events landed within 60 minutes of emission.
• SLO: 99.5% per week.
• Error budget: 0.5% weekly.
• Sev policy: Sev-2 if weekly projection shows budget breach; Sev-1 if slice affecting checkout events > 0.7% for > 2 hours.
• Ownership: Platform data ingestion team Accountable; AE Consulted for downstream model lag.

Example 3: Accuracy via reconciliation

• SLI: absolute % variance of revenue vs billing system for prior day.
• SLO: |variance| ≤ 0.5% for 95% of days.
• Controls: automated recon report; sample manual audit monthly.
• Action: if |variance| > 0.5% then downgrade dashboards and notify Finance.

Templates and snippets

Dataset: [name]
Criticality: [High/Medium/Low]
Business use: [who uses it, for what]
SLIs: [metric name + definition]
SLOs: [target + window]
Error budget: [allowed misses]
Severity & response: [Sev-1/2/3: response & resolution times]
Ownership (RACI): [Responsible, Accountable, Consulted, Informed]
Monitoring: [where metrics/alerts live]
Exceptions: [process, approver, end date]
Review: [cadence]

Hands-on exercises

These mirror the exercises below. Try them now. The quick test is open to everyone; saving your progress is available to logged-in users.

1. Draft an SLA: Pick a dataset and fill the minimal SLA template. Include one freshness SLI and one completeness SLI.
2. Map ownership (RACI): For one pipeline, assign R, A, C, I for normal ops and for incidents.
3. Write a runbook: For a missed freshness SLO, write 6–8 bullet steps from alert to recovery.

• Checklist: SLIs are measurable; thresholds are realistic; error budget defined; response times clear; owners named; exception window rule exists.

Common mistakes and self-check

• Vague metrics: “data is up to date.” Self-check: can a script compute it?
• Targets without baseline. Self-check: did you analyze 2–4 weeks of historical performance?
• No error budget. Self-check: do you know how many misses trigger corrective action?
• Alerting to everyone or no one. Self-check: is there exactly one Responsible on-call?
• Ignoring schema changes. Self-check: do you have a notice period and compatibility checks?
• One-size-fits-all severity. Self-check: does severity depend on business impact time?

Practical projects

• Create SLAs for three datasets of different criticality and implement automated SLI checks and alerts.
• Build a reconciliation job comparing a key metric to its source-of-truth and gate downstream publishes when it fails.
• Run an incident drill: simulate a missed freshness SLO, execute the runbook, and document learnings.

Who this is for

• Analytics Engineers and BI Developers who publish models used for decisions.
• Data Stewards and Product Analysts who define data expectations.

Prerequisites

• Basic understanding of your data platform’s scheduling and monitoring.
• Ability to write simple tests/queries to compute SLIs (e.g., row counts, time lags).

Learning path

• First: define SLIs for one critical dataset.
• Next: agree SLOs and error budget with stakeholders.
• Then: implement alerts and a runbook, and practice an incident drill.
• Finally: scale with templates, RACI, and a quarterly SLA review.

Mini challenge

Your marketing attribution model often completes by 06:20, but twice per month finishes at 06:50 due to upstream API limits. Finance wants 06:30 hard cutoffs; Marketing prefers occasional delays to avoid partial data. Draft an SLO and error budget that balances both, set severity thresholds, and outline who approves temporary exceptions.

Next steps

• Apply the SLA template to one dataset this week.
• Schedule a 30-minute review with the dataset’s business owner.
• Take the quick test below to confirm understanding.