Documentation

Why documentation matters for BI Analysts

Documentation turns your knowledge into a durable, shared asset. As a BI Analyst, great docs help teams trust metrics, debug faster, onboard stakeholders, and keep dashboards accurate through change. It unlocks smoother releases, fewer ad-hoc questions, and stronger decision-making.

• Reduce ambiguity: one source of truth for metrics and definitions.
• Speed up delivery: reusable notes and runbooks cut investigation time.
• Increase adoption: clear usage notes help stakeholders read dashboards correctly.
• De-risk changes: change logs and lineage make impact easy to assess.

Who this is for

• BI Analysts who own dashboards, KPIs, and reporting processes.
• Data/Analytics Engineers collaborating on models and pipelines.
• Team leads who need reliable, auditable decision support.

Prerequisites

• Basic SQL (SELECT, JOIN, GROUP BY) and comfort reading query plans at a high level.
• Familiarity with your BI tool (for example: dashboard filters, data refresh schedules).
• Understanding of your organization’s core KPIs and data sources.

Learning path

1. Start with a Metric Glossary and Data Dictionary to standardize definitions.
2. Map Source → Model → Dashboard lineage so changes are traceable.
3. Capture modeling and transformation notes to explain the logic behind KPIs.
4. Write dashboard usage notes to prevent misinterpretation.
5. Introduce change and decision logs to track what changed and why.
6. Create runbooks for incidents and common data issues.
7. Build onboarding guides to help stakeholders self-serve.
8. Keep docs aligned with releases using a lightweight checklist.

Practical roadmap

1. Pick a home for docs (e.g., a shared folder or wiki). Define a simple structure: Glossary, Dictionary, Lineage, Modeling Notes, Dashboard Notes, Logs, Runbooks, Onboarding.
2. Document the top 5 KPIs first (owner, calculation, constraints, refresh).
3. Outline lineage for 1 critical dashboard: source tables → transformations → datasets → dashboard tiles.
4. Add modeling notes for the queries/models feeding those KPIs.
5. Write dashboard usage notes and common pitfalls.
6. Start change and decision logs and backfill recent changes.
7. Create one runbook for a frequent issue (e.g., “stale data”).
8. Publish a 1-page onboarding guide for new stakeholders.
9. Adopt a release-aligned checklist to keep docs current.

{
  "name": "Active Users (7-day)",
  "owner": "Analytics",
  "business_definition": "Unique users with at least one session in the last 7 days.",
  "sql_definition": "COUNT(DISTINCT user_id) over last 7 days",
  "time_grain": "daily",
  "filters": ["exclude internal users"],
  "refresh_cadence": "daily 06:00 UTC",
  "edge_cases": ["bots filtered", "merged accounts"],
  "quality_checks": ["rowcount > 0", "no >20% drop DoD"],
  "dashboards": ["Product Health"]
}

2025-02-12 | Dashboard: Product Health | Change: Updated Active Users filter to exclude test devices | Impact: ~-2% | Owner: BI | Related Decision: DEC-014

DEC-014 | 2025-02-10 | Topic: Exclude test devices from Active Users | Options considered: keep vs. exclude | Decision: Exclude | Rationale: better represent real usage | Review date: 2025-05-10

Incident: Dashboard shows stale data
Detect: refresh alert, user report, or 0-row tile
Check: pipeline status → model freshness → BI extract time
Fix: rerun model; if fails, rollback last change
Communicate: post status, ETA, and impact
Prevent: add freshness test, update monitoring

Worked examples

Metric: Conversion Rate (Signup → Purchase)
Owner: Growth Analytics
Business definition: Purchases / Signups within 7 days of signup
SQL sketch:
SELECT
  COUNT(DISTINCT CASE WHEN purchased_within_7d THEN user_id END) * 1.0
  / NULLIF(COUNT(DISTINCT user_id), 0) AS conv_rate
FROM signup_cohort
-- Notes: exclude employees; deduplicate by user_id
Edge cases: refunds not excluded; guest checkouts excluded today
Quality checks: conv_rate between 0 and 1; day-over-day change < 10pp
Dashboards: Growth Funnel

Table: fact_orders
- order_id: STRING | PK | unique order identifier
- user_id: STRING | FK → dim_users.user_id | PII: yes
- order_ts: TIMESTAMP | UTC | ingestion lag < 5m
- status: STRING | enum[pending, paid, refunded, canceled]
- total_amount: DECIMAL(12,2) | currency=USD | taxes included
Notes: Source is payments_api v2. Missing tax in v1 (before 2023-01-01).

Source: payments_api.transactions
Transform 1: stg_transactions (clean types)
Transform 2: fct_orders (aggregated payments)
Semantic: metrics.orders_total
Dashboard: Revenue Overview → tile: MRR trend
Validation: rowcount parity between stg and source; sum(total) within ±1% of finance report

Model: fct_orders
Purpose: one row per order
Primary keys: order_id
Logic highlights: joins stg_transactions + stg_refunds; excludes test merchants
Tests: unique(order_id), not_null(order_id), freshness(order_ts)
Performance: materialized as table, daily full refresh
Breaking changes: changing status mapping will affect Revenue Overview

Symptom: Conversion Rate dropped to 0
1) Check data freshness for signup and purchase tables
2) Validate joins (user_id changes?)
3) Run last known-good query manually
4) If schema changed, update model + dictionary, log change
5) Communicate impact and ETA

Dashboard: Product Health
Audience: Product managers, leadership
How to read: 28-day rolling average; filters apply to all tiles
Common pitfalls: segment filters reset on refresh; excludes internal traffic
When to use: weekly product review; not for daily operational decisions

Drills and exercises

• [ ] Write a glossary entry for one KPI, including SQL sketch and edge cases.
• [ ] Document lineage for a dashboard tile (source → transforms → tile).
• [ ] Add three data dictionary fields with types, constraints, and notes.
• [ ] Create a runbook for “stale dashboard” or “duplicate rows.”
• [ ] Backfill two recent changes into your change log with dates and impacts.
• [ ] Draft onboarding notes for a new stakeholder: when to use which dashboard.

Common mistakes and debugging tips

• Mixing business and SQL definitions without clarity. Provide both and tag which is authoritative.
• Omitting edge cases. List exclusions, null handling, and known gaps.
• Stale docs after releases. Tie doc updates to your release checklist.
• Ambiguous ownership. Every metric and dashboard should have an owner.
• No impact notes in change logs. Always include estimated impact and affected assets.
• Runbooks without detection steps. Start with how to detect the issue before fixing it.

Mini project: KPI documentation pack

Deliver a complete documentation pack for one KPI that appears on a key dashboard.

• Metric glossary entry with SQL sketch, edge cases, owner, refresh.
• Data dictionary fields for all involved tables.
• Source → model → dashboard lineage map.
• Modeling/transformation notes for the main dataset.
• Dashboard usage notes and common pitfalls.
• Change log entry for the most recent update.
• Runbook for one likely incident impacting this KPI.

Subskills

• Data Dictionary And Metric Glossary — standardize terms, fields, and KPIs so everyone speaks the same language.
• Source To Dashboard Lineage — trace data from origin to tiles for impact analysis.
• Dashboard Usage Notes And Definitions — prevent misreads and explain when/how to use a dashboard.
• Modeling And Transformation Notes — capture logic, tests, and assumptions behind datasets.
• Change Log And Decision Log — record what changed and why for auditability.
• Runbooks For Incidents And Data Issues — enable fast detection, triage, and recovery.
• Onboarding Guides For Stakeholders — help newcomers self-serve with confidence.
• Keeping Docs Updated With Releases — align documentation updates with your release process.

Practical projects

• Create a “Top 5 KPIs” glossary with SQL sketches and link each to dashboards.
• Write a runbook library for two recurring issues (stale data, duplicate rows).
• Build an onboarding one-pager: where to find metrics, refresh cycles, and who owns what.

Next steps

• Adopt the templates above and ship the mini project for one KPI this week.
• Expand coverage: add two KPIs and one dashboard per week.
• Integrate docs into your release checklist so updates always ship with changes.