Why this matters
- Checkpoints: tags/releases (e.g., v1.2.0).
- Small steps: small PRs/commits with clear messages.
- Safety rope: reversible changes (migrations with up/down), feature flags, backups/exports.
Quick toolkit (no custom tooling needed)
- Versioning: use tags (e.g., v1.2.1) when promoting to Prod.
- Releases: merge to main, run tests, tag, deploy.
- Rollback options:
- Code: git revert <commit> (creates a new commit that undoes changes).
- File-level: git restore --source <tag> path/to/file.sql.
- Environment: redeploy previous tag or import previous BI artifact (PBIP/PBIT/TDS/LookML branch).
- Data: run down migration or restore snapshot/backup.
- Gradual rollout: Dev → Test → Prod, plus feature flags or limited audience.
Safe release principles for BI
- Make changes reversible: pair every schema change with a down path.
- Tag every production release: creates a clear rollback target.
- Keep changes small and isolated: easier testing, faster rollback.
- Deploy config with code: metric definitions, SQL logic, and dashboard files under version control.
- Verify with checks: row counts, summary metrics, spot checks on critical dashboards.
- Communicate: change notes and who’s on point for rollback.
Worked examples
Example 1: Revert a breaking SQL change
Scenario: You changed a revenue calculation in metrics.sql, and finance reports spike unexpectedly.
- Identify the last known good tag: v1.1.0.
- Compare diff: git diff v1.1.0..main -- metrics.sql
- Rollback fast (file-level): git restore --source v1.1.0 metrics.sql; commit with message "revert: restore metrics.sql from v1.1.0"
- Redeploy and validate key KPI checks.
Why this works
Restoring from a tag makes a clear, auditable revert commit. You don’t rewrite history; you add a safe undo step.
Example 2: Rollback with git revert
Scenario: A single commit introduced a slow join in model_orders.sql.
- Find commit: git log --oneline -- model_orders.sql
- Revert: git revert <bad_commit_sha>
- Push, deploy, and watch performance metrics.
Tip
git revert creates a new commit that undoes the bad commit without force-pushing.
Example 3: Safe release with tag and hotfix
Scenario: You are releasing v2.0.0 with a new dimension. You want a fast rollback path.
- Merge to main, run tests, tag: git tag -a v2.0.0 -m "release: v2.0.0"; git push --tags
- Deploy v2.0.0 to Test; verify dashboard filters and row counts.
- Promote to Prod.
- Issue found? Rollback by redeploying v1.9.2: git checkout v1.9.2; deploy artifacts; announce rollback.
- Patch hotfix branch from main; release v2.0.1 after fix.
Notes
- Tags are your fast rollback anchors.
- Always write a one-line change note for each tag.
Release checklist (tick before Prod)
- All SQL/model changes have down/undo scripts or a documented revert path
- Tag planned (e.g., v1.3.0) and change notes drafted
- Test environment deployed; KPIs verified (row counts, key sums, null checks)
- Backup/snapshot or export of the previous artifact exists
- On-call/owner named; rollback steps written
- Small blast radius first (limited audience or phased rollout) where possible
Common mistakes and self-check
- Mistake: Force-pushing to rewrite history to “hide” a bad commit. Fix: Use revert and document the reason.
- Mistake: No tag for the last good version. Fix: Tag every Prod deploy.
- Mistake: Irreversible schema migrations. Fix: Always write down migration or a safe path (e.g., create new column, backfill, then swap).
- Mistake: Deploying big batches of changes. Fix: Ship small and testable units.
- Mistake: Skipping validation. Fix: Predefine 3–5 KPI checks per release.
Self-check prompts
- Can I point to the exact tag I’d redeploy if needed?
- Is my rollback path documented and under 10 minutes?
- What’s the small-scope test that proves success before full rollout?
Exercises (hands-on)
Available to everyone. Only logged-in users get saved progress.
- Exercise 1 — Revert a breaking change with a new commit
Mirror of exercise ex1 below.
- Create a demo repo; make a bad change in metrics.sql; commit.
- Use git revert to undo just that commit; push.
- Tag a patch release v1.1.1 and write a one-line change note.
- Exercise 2 — Write a safe release and rollback plan
Mirror of exercise ex2 below.
- Draft a step-by-step plan to release a new dimension table.
- Include validation checks, a tag, and a rollback path in under 10 minutes.
Mini challenge
Your new calculation adds a CASE statement that could misclassify 1% of orders. Describe a safe phased rollout using Test → limited audience → Prod, with a tag at each step and a one-command rollback.
Possible approach
- Deploy to Test; validate with sample KPIs.
- Create Prod release tag v3.0.0-rc; publish to a limited app/workspace audience.
- Monitor metrics; if healthy, promote to v3.0.0.
- Rollback path: redeploy v2.9.1 tag or revert commit; announce status.
Who this is for
- BI Analysts shipping SQL models, dashboards, or metric definitions.
- Analysts contributing to versioned analytics repos.
Prerequisites
- Basic Git: clone, pull, commit, branch, merge.
- Comfort editing SQL models or BI project files.
Learning path
- Branching and commits basics.
- Tags and releases.
- Safe rollout patterns (this lesson).
- Monitoring and validation after deploy.
Practical projects
- Build a sample analytics repo with a metrics.sql and a dashboard definition; practice tagging and rolling back.
- Create a reversible migration (add column + backfill + down script) and test end-to-end.
Next steps
- Automate a small validation script (row counts, key sums) to run after each deploy.
- Adopt a simple release note template for every tag.