SLO SLA Concepts

Who this is for

• Backend and platform engineers who operate APIs, services, or data systems.
• Developers introducing reliability goals to a new or growing product.
• Engineers preparing for on-call, incident response, or SRE collaboration.

Prerequisites

• Basic understanding of HTTP services and deployment environments.
• Familiarity with logs/metrics and a monitoring tool (any is fine).
• Comfort with percentages and simple time calculations.

Why this matters

SLOs and SLAs turn reliability into measurable engineering work. They guide what to monitor, when to alert, and how to trade off shipping speed vs stability.

• Set clear targets (e.g., 99.9% availability, p95 latency ≤ 200 ms).
• Decide when to page vs when to fix during work hours.
• Align product promises (SLA) with realistic engineering capability (SLO).

Concept explained simply

• SLI (Service Level Indicator): The measurement. Example: percentage of successful requests; p95 latency.
• SLO (Service Level Objective): The target for the SLI. Example: 99.9% success per month; p95 latency ≤ 200 ms.
• SLA (Service Level Agreement): An external promise (often contractual) to customers. Usually includes credits/penalties.

Error budget = 100% − SLO. If SLO is 99.9% availability for a month, the error budget is 0.1% downtime for that month.

Worked examples

1) Monthly downtime allowed for 99.9% availability

30-day month has 43,200 minutes. Error budget for 99.9% = 0.1% of time.

• 0.1% of 43,200 minutes = 43.2 minutes = 43 minutes 12 seconds.
• If you had a 20-minute outage, about 46% of your monthly budget is already gone.

2) Error budget by request count

SLO: 99.5% request success rate per month. Total requests this month: 10,000,000.

• Error budget = 0.5% of 10,000,000 = 50,000 allowed failed requests.
• At 35,000 failures you have consumed 70% of the budget; consider slowing risky changes.

3) Multi-window burn-rate alerting

SLO: 99.9% availability (monthly). Monthly error budget = 0.1% of time ≈ 43m12s.

• Fast burn page: If in the last 1 hour you burn ≥ 2% of the monthly budget, page immediately. That’s ≈ 0.864 minutes (51.8 seconds) of bad time in 1 hour.
• Slow burn ticket: If in the last 6 hours you burn ≥ 5% of the monthly budget, create a ticket. That’s ≈ 2m9.6s of bad time in 6 hours.
• This catches both acute incidents and smoldering issues.

4) SLA vs SLO alignment

If your internal SLO is 99.9%, avoid promising a stricter external SLA like 99.99% unless you have evidence you consistently exceed 99.99% with margin. A safer SLA might be 99.9% or 99.95% while you improve.

How to choose SLO targets (step-by-step)

1. Identify critical user journeys: login, search, checkout, publish, etc.
2. Pick SLIs per journey: availability, success rate, p95 latency, freshness, throughput.
3. Look at current performance: last 30–90 days; find p95/p99, success percentile.
4. Set realistic SLOs: slightly better than current median performance but not fantasy. Example: If p95 ≈ 240 ms, set SLO at ≤ 250 ms before aiming lower.
5. Define the window: monthly is common for availability; weekly can fit fast-moving teams.
6. Add error budget policies: what happens when budget is low (freeze risky deploys, prioritize reliability work).
7. Define alert rules: multi-window burn-rate pages and tickets; minimize noisy alerts.

Implementation checklist

• SLIs are precisely defined (what counts as good vs bad and any filters).
• SLO targets have a clear window (e.g., rolling 28–30 days).
• Error budget is computed and visible to the team.
• Burn-rate alerts exist for fast and slow windows.
• Runbooks: what to check first, who to page, rollback steps.
• Post-incident reviews update SLOs and alerts when needed.

Practice exercises

These mirror the exercises below so your answers can be checked. A quick test is also available to everyone; only logged-in users get saved progress.

Exercise 1: Downtime budget + alert thresholds

SLO: 99.95% monthly availability. Calculate the monthly downtime budget, then propose two burn-rate-based alerts: a fast page (≈2% budget in 1 hour) and a slow ticket (≈5% in 6 hours). Show your math.

Exercise 2: Latency SLO by percentile

SLO: p95 latency ≤ 200 ms over 1,000,000 requests. This month, 88,000 requests exceeded 200 ms. Did you meet the SLO? Explain.

Self-check checklist

• I computed downtime/error budgets correctly for the time window.
• I converted percentages to minutes/seconds or counts precisely.
• I used percentile thinking for latency (allowed tail size = 5% for p95).
• My alert thresholds consume a meaningful fraction of the budget, not noise.

Common mistakes and how to self-check

• Mixing SLA and SLO: Promising customers your internal stretch goal. Self-check: Is your external promise ≤ your proven reliability?
• Using averages for latency: Averages hide tail pain. Self-check: Are you tracking p95/p99?
• Over-alerting: Alert on every blip. Self-check: Are alerts budget-aware with multi-window burn rates?
• Too strict too soon: Setting 99.99% without evidence. Self-check: Do historical metrics show consistent headroom?
• Undefined "good": Ambiguous SLIs. Self-check: Is success clearly defined (e.g., HTTP 2xx within 200 ms)?

Practical projects

• Project A: SLO Scorecard — Build a spreadsheet that ingests weekly metrics and computes success rate, p95 latency, and error budget burn. Include conditional formatting when budget < 30%.
• Project B: Synthetic + Real SLI — Set up a simple health endpoint, a basic synthetic check, and a dashboard that combines synthetic uptime with real request success rate.
• Project C: Burn-rate Alerts — Implement two alerts for 99.9% availability: fast page (≈2% budget in 1h) and slow ticket (≈5% in 6h). Add a short runbook.

Mini challenge

You operate a write API with two key user journeys: Create and Update. Pick one SLI for each (availability or latency), set an SLO target and window, and write one sentence explaining why your choice best reflects user impact.

Learning path

• Before: Monitoring basics (metrics, logs, percentiles), incident response fundamentals.
• Now: SLI/SLO/SLA definitions, error budgets, burn-rate alerting.
• Next: Capacity planning, load testing, redundancy and graceful degradation.

Next steps

• Define one SLI and SLO for your most important endpoint this week.
• Publish a one-page SLO doc and share it with your team.
• Set one fast-burn and one slow-burn alert tied to your error budget.