Who this is for
- BI Analysts who maintain dashboards, metrics, or data models.
- Analysts working with stakeholders who ask “what changed and why?”
- Anyone expected to justify data or reporting decisions later.
Prerequisites
- Basic understanding of your analytics stack (dashboards, data models, metrics).
- Comfort writing concise, factual notes.
- Access to a shared documentation space (wiki, README, or analytics repo).
Why this matters
In BI, things change: metric definitions, data sources, transformations, dashboards. Without a record, teams lose context and repeat debates.
- Audits and trust: Stakeholders can see when and why a number changed.
- On-call analytics: When a metric looks off, you can quickly check recent changes and roll back if needed.
- Knowledge transfer: New team members learn the reasoning behind past decisions, not just the outcomes.
- Faster approvals: Clear decision logs reduce back-and-forth by showing options considered and trade-offs.
Concept explained simply
Change Log = a chronological list of changes made to data assets (what changed, when, who, where, and impact).
Decision Log = a simple record of why a choice was made (problem, options, decision, reasoning, owner, date).
Mental model
Think of the change log as a “flight log” that notes each adjustment during the journey, and the decision log as the “pre-flight briefing” that documents why a particular route was chosen.
Simple templates you can copy
Change Log template
Change ID: CL-YYYYMMDD-##
Date/Time (UTC):
Owner:
Assets impacted: (dashboards, models, metrics)
Summary: (1–2 lines)
Details: (what changed, where)
Reason: (why we did it)
Impact/Risk: (user impact, data impact)
Validation: (how verified)
Rollback plan: (if needed)
Status: (planned | in-progress | done | rolled-back)Decision Log template
Decision ID: DL-YYYYMMDD-##
Date:
Owner/Approver:
Context/Problem:
Options considered: (A, B, C)
Decision: (chosen option)
Rationale: (trade-offs, constraints)
Impacted areas: (teams, metrics, dashboards)
Review date (optional): (when to revisit)
References: (ticket IDs, related change IDs)Worked examples
Example 1 — Metric rename with calculation change
Change ID: CL-20240115-01
Date/Time (UTC): 2024-01-15 10:00
Owner: A. Rivera
Assets impacted: Metric: Active Users (DAU); Dashboard: Growth KPIs
Summary: Renamed "Daily Active Users" to "Active Users (DAU)" and updated calculation.
Details: Calculation now excludes internal test accounts using user.flag_internal = false.
Reason: Reduce noise; align with product definition approved last week.
Impact/Risk: ~4–6% lower DAU; stakeholder communications required.
Validation: Compared last 28 days old vs new calc; variance matches expectations.
Rollback plan: Revert filter in metric definition; restore old title.
Status: doneExample 2 — Switching refresh schedule
Change ID: CL-20240203-02
Date/Time (UTC): 2024-02-03 07:30
Owner: K. Lee
Assets impacted: Model: sales_orders; Dashboard: Sales Overview
Summary: Changed refresh from hourly to every 15 minutes.
Details: Adjusted scheduler; incremental model now processes new partitions only.
Reason: Sales requested fresher order figures for midday standups.
Impact/Risk: Slightly higher warehouse cost; improved latency.
Validation: 1-day trial with monitoring; no failures, cost +8% within budget.
Rollback plan: Revert scheduler to hourly.
Status: doneExample 3 — Decision: Revenue definition
Decision ID: DL-20240310-01
Date: 2024-03-10
Owner/Approver: BI Lead
Context/Problem: Discrepancies between "Gross" and "Net" revenue across reports.
Options considered: (A) Report Net only; (B) Report both with clear labels; (C) Keep current mixed definition.
Decision: (B) Report both metrics with explicit labels and tooltips.
Rationale: Transparency avoids confusion; stakeholders need both views.
Impacted areas: Finance dashboards, Product revenue KPIs.
Review date: 2024-06-10
References: Ticket FIN-231, Change IDs CL-20240311-01, CL-20240312-02How to write solid entries
- Start with a short, specific summary. Avoid vague phrases.
- State where the change lives (model name, metric name, dashboard).
- Explain why now (trigger, request, incident).
- Estimate impact and risk; include how you validated.
- Always include a simple rollback plan.
- Cross-reference: link Decision IDs to related Change IDs (plain text IDs are enough).
- I used clear asset names
- I wrote a one-line summary
- I documented impact/validation
- I added a rollback plan
Common mistakes and self-check
1) Logging the change but not the reason
Fix: Add one sentence of rationale and the triggering request or issue.
2) No impact or validation
Fix: Write how you measured success or confirmed no regression (e.g., backfill comparison, sample checks).
3) Mixing decisions and changes
Fix: Decisions capture choices among options; changes capture implementation details. Keep both and cross-reference IDs.
4) Titles too vague
Fix: Include asset + action: "Rename metric X; exclude internal users" is better than "Metric update".
Self-check before publishing
- Could someone debug a dashboard with only this entry?
- Would a new hire understand why this was done?
- Can we roll it back using what’s written?
Exercises
Do these in your doc tool or a plain text file. Compare with the sample solutions below.
Exercise 1 — Write a Change Log entry
Scenario: The "Leads by Source" dashboard started double counting leads because the CRM added a new source type "partner_referral" that maps to two tables. You fixed the model to de-duplicate by email + created_at within 24 hours.
- Include assets, summary, details, reason, impact, validation, rollback.
- Give it an ID and owner.
Show solution idea
Change ID: CL-20240421-01
Date/Time (UTC): 2024-04-21 16:20
Owner: J. Patel
Assets impacted: Model: crm_leads; Dashboard: Leads by Source
Summary: Fixed double counting for partner_referral leads.
Details: De-duplicate on lower(email) + date_trunc('day', created_at); added source normalization.
Reason: CRM introduced new source type causing duplicates.
Impact/Risk: Past 7 days leads drop ~3%; attribution more accurate.
Validation: Compared last 14 days with CRM exports; dedupe count matches.
Rollback plan: Revert dedupe logic commit; keep snapshot for quick restore.
Status: doneExercise 2 — Write a Decision Log entry
Scenario: For churn, you must choose between (A) last-activity-based churn or (B) subscription-cancellation-based churn. Product wants faster signal; Finance wants auditable numbers.
- Capture context, options, decision, rationale, impacted areas, review date.
- Add references to any related changes (use placeholder IDs).
Show solution idea
Decision ID: DL-20240505-02
Date: 2024-05-05
Owner/Approver: Analytics Manager
Context/Problem: Need a consistent churn metric for weekly reporting and board packs.
Options considered: (A) Activity-based churn (no activity for 28 days); (B) Cancellation-based churn (subscription ended).
Decision: (B) Use cancellation-based churn for official reporting; (A) as leading indicator in product analytics.
Rationale: Finance needs auditable, contractual definition; product still benefits from early signal.
Impacted areas: Finance KPIs, Growth dashboards, Lifecycle experiments.
Review date: 2024-08-05
References: Related Change IDs CL-20240506-01, CL-20240506-02- I used the correct template for each exercise
- I wrote a clear summary line
- I included impact and validation
- I added cross-references (IDs)
Practical projects
- Backfill a Change Log: Review the last 4 significant analytics changes you made. Write proper entries for each using the template.
- Start a Decision Register: Create a page that lists 10 key analytics decisions. Add review dates and owners.
- Incident to Log: Take a past data incident (bad refresh, wrong metric). Write both a Decision Log (fix choice) and a Change Log (implementation).
Learning path
- Understand the difference between changes and decisions.
- Adopt the templates above and customize field names to your team.
- Log every change for 2 weeks; review quality weekly.
- Introduce a weekly Decision Review: add new decisions, revisit old ones.
- Automate reminders (calendar/task tool) for review dates.
Mini challenge
In three sentences, write a Decision Log entry choosing between a complex but precise metric vs. a simpler proxy for a beta dashboard. Include context, decision, and rationale.
See a compact answer
Context: Beta dashboard needs a revenue signal; precise metric requires long backfill. Decision: Use proxy (orders * AOV) for beta, precise metric for GA. Rationale: Faster iteration now; accuracy later before formal reporting.
Next steps
- Pick one live dashboard and start logging all changes for the next sprint.
- Create a shared Decision Register and post 3 current analytics decisions.
- Take the Quick Test below to check understanding. Everyone can take the test; only logged-in users have their progress saved.