Change Tracking For Models And Reports

Why this matters

As a BI Developer, you change data models (tables, columns, relationships, measures) and report definitions (visuals, fields, filters). Without disciplined change tracking you risk broken dashboards, lost work, and production outages. Teams expect repeatable releases, auditability, and fast rollback.

• Real tasks: add a KPI, rename a column, deprecate a measure, update default filters, migrate a model to a new schema, and release changes safely.
• Your goals: make each change visible, reviewable, testable, reversible.

Concept explained simply

Change tracking is the practice of recording every model/report change in source control with clear intent, diffs, tests, and release notes so anyone can reproduce and rollback.

Benefits: fewer surprises, faster reviews, easy rollback, better collaboration.

Mental model

Think in contracts:

• Schema contract: table/column names and data types.
• Metric contract: measure names and calculations.
• Query/semantic contract: relationships, permissions, partitions.
• Report contract: fields bound to visuals, slicers, filters, bookmarks.

Any change can be additive, modifying, or destructive. Use semantic versioning: MAJOR.MINOR.PATCH.

• MAJOR: breaking changes (rename/remove columns/measures used in production).
• MINOR: backward-compatible additions (new columns/measures/visuals).
• PATCH: fixes that don’t change contracts (typos, formatting, comment-only refactors).

Change types and impact

• Additive (usually safe): new column, new measure, new visual/page.
• Modifying (risk): datatype change, measure logic change, relationship change, default filter change.
• Destructive (breaking): rename/remove column/measure referenced by reports, drop table, remove role.

A safe change-tracking workflow

1. Create a feature branch named after intent (e.g., feature/add-revenue-measure or breaking/rename-customer-id).
2. Write a short change plan (what, why, risk, rollback).
3. Make the model/report change in source-controlled text formats (model JSON/YAML, TMSL/TMDL, LookML, dbt SQL/YAML, PBIP or report JSON exports).
4. Validate locally: refresh tests, run queries, open reports to check visuals.
5. Update tests/snapshots and add deprecation notices when needed.
6. Commit with a precise message and a small, focused diff.
7. Open a pull request with a checklist (impact, tests, screenshots, migration steps).
8. Merge, tag a version (MAJOR.MINOR.PATCH), deploy, and monitor.

Worked examples

Example 1: Add a new KPI (non-breaking)

Change: introduce measure Revenue = SUM(fact_sales.amount).

ADD: new measure Revenue in fact_sales

- model/semantic.json: add measure Revenue
- tests/metrics.test.yml: add threshold checks
- reports/ExecutiveOverview.json: add card visual bound to [Revenue]

SemVer: MINOR

Example 2: Rename a column (breaking)

Change: dim_customer.customer_id to customer_key. Reports reference customer_id.

Example 3: Change default report filter (modifying)

Change: default date range from last 30 days to last 90 days. Not a schema break, but affects numbers and alerts.

Hands-on exercises

Note: The quick test is available to everyone; only logged-in users will have their progress saved.

Exercise 1: Track a model change with semantic versioning

Scenario: Your model currently defines a measure Sales = SUM(fact_sales.amount). Product wants Revenue that excludes refunds. You will:

• Plan the change and its impact.
• Add a new measure Revenue = SUMX(fact_sales, fact_sales.amount - fact_sales.refund_amount).
• Decide version bump.
• Write a commit message and a change log entry.

- Change type: Additive (new measure)
- Version: MINOR
- Commit: ADD: new [Revenue] measure excluding refunds; tests + report tile
- Changelog entry with purpose, tests updated, affected report page

Commit:
ADD: [Revenue] excluding refunds; add KPI tile and metric tests

Changelog:
- Why: Business needs net revenue distinct from gross sales
- What: Added [Revenue] = SUMX(fact_sales, amount - refund_amount)
- Impact: Non-breaking addition
- Tests: Metric sanity check within expected range; report screenshot added
Version: 1.4.0 - MINOR

Exercise 2: Diff a report change and flag breaking impacts

Before: a bar chart uses field dim_product.category and measure [Sales]. After: visual uses dim_product.category_name; [Sales] replaced by [Revenue].

• List diffs you would expect in version control.
• Which changes are breaking? What version bump?
• What tests or screenshots would you add?

- Diffs: field binding change (category -> category_name), measure binding change (Sales -> Revenue), title/legend updates
- Breaking? If category_name is a new additive column and Sales still exists, report change is not breaking; model remains compatible
- Version: MINOR (behavioral change)
- Add screenshots, regression check for visual

Pre-PR checklist

• Change type labeled (Additive/Modifying/Breaking).
• Commit messages clear and scoped.
• Tests updated; local validation passed.
• Screenshots for report UIs included.
• Version bump decided; changelog updated.
• Rollback plan documented.

Common mistakes and self-check

• Silent renames: Renaming a column/measure without a deprecation period. Self-check: search references across model and reports.
• Big bang commits: Mixing schema, metrics, and visuals in one commit. Self-check: each commit should explain one intent.
• No screenshots: Reviewers can’t see UI impact. Self-check: attach before/after of affected pages.
• Skipping tests: Metrics change but tests don’t. Self-check: run/record test results in PR.
• Wrong version bump: Using PATCH for behavior changes. Self-check: use MAJOR/MINOR/PATCH rules strictly.

Practical projects

• Introduce a new dimension attribute across model and two reports. Track all changes, tests, screenshots, and release as a MINOR bump.
• Run a deprecation: add a replacement measure, migrate visuals, then remove the old measure in a MAJOR release.
• Create a change log site or page from commits and tags. Include migration notes per version.

Learning path

1. Learn semantic versioning for BI assets.
2. Adopt text-based representations for models/reports.
3. Practice small, single-intent commits with clear messages.
4. Add tests/snapshots to validate metrics and visuals.
5. Use PR checklists and release tags consistently.

Who this is for

• BI Developers managing datasets, metrics, and reports.
• Analytics Engineers maintaining semantic layers and dashboards.

Prerequisites

• Basic Git: branching, committing, pull requests.
• Familiarity with your BI tool’s model/report artifacts.
• Ability to run local validations/tests.

Next steps

• Create a team-wide PR template with change type and version bump sections.
• Add deprecation guidelines (minimum 1–2 release cycles).
• Automate screenshots and metric checks where possible.

Mini challenge

Your dataset is used by 200 users. You need to rename dim_order.order_id to order_key. Propose a no-downtime plan in 5 steps, including commit sequence, PR checklist notes, and when to do the MAJOR bump.