Who this is for
Mental model
- Think user journey: source data → transform → query → visualize → interpret
- Review from risk-first: correctness and privacy before style
- Comment to help the next person understand and extend the work
BI/Viz Review Checklist (use as you read the PR)
- Scope & clarity
- PR title/description explain the business goal and change
- Linked issue or clear context is included in the description
- Correctness & data integrity
- Metric definitions match business logic
- Filters, joins, and time grains are explicit and correct
- Nulls, outliers, and timezone handled
- Visualization accuracy
- Axis scales, labels, legends, and units correct
- Color encodings accessible and consistent
- Defaults documented (e.g., date range, segments)
- Performance
- SQL uses appropriate indexes/aggregations
- Heavy calculations pushed down to warehouse when possible
- Datasets sampled for preview; full queries benchmarked
- Style & maintainability
- Names are descriptive and consistent
- Code/spec is modular and commented where non-obvious
- Dead code removed; files organized
- Version control hygiene
- Small, focused PRs; meaningful commit messages
- No secrets/keys; environment-safe
- Tests/checks pass; screenshots attached when UI changes
- Reproducibility & handoff
- Steps to run locally; seeds/mocks included if needed
- Change log or doc snippet updated
Worked examples
Example 1 — Review a KPI SQL change
Open SQL snippet
-- goal: monthly revenue per product (net of refunds)
SELECT
DATE_TRUNC('month', o.order_date) AS month,
p.product_name,
SUM(o.amount) AS revenue
FROM orders o
JOIN products p ON o.product_id = p.id
WHERE o.status = 'completed'
GROUP BY 1,2
ORDER BY 1,2;- Correctness: Are refunds excluded? If refunds exist in a separate table, net revenue should subtract them.
- Time grain: Confirm timezone alignment; DATE_TRUNC on UTC vs business timezone.
- Joins: Check for product deletions; LEFT JOIN may be safer if historical orders exist for retired products.
- Performance: Consider pre-aggregating or adding a materialized view if used widely.
- Documentation: PR description should note “net of refunds” and how refunds are handled.
Better version
WITH revenue AS (
SELECT DATE_TRUNC('month', order_date AT TIME ZONE 'UTC') AS month,
product_id,
SUM(amount) AS gross
FROM orders
WHERE status = 'completed'
GROUP BY 1,2
), refunds AS (
SELECT DATE_TRUNC('month', refund_date AT TIME ZONE 'UTC') AS month,
product_id,
SUM(refund_amount) AS refunded
FROM order_refunds
GROUP BY 1,2
)
SELECT r.month,
p.product_name,
COALESCE(r.gross,0) - COALESCE(f.refunded,0) AS net_revenue
FROM revenue r
LEFT JOIN refunds f USING (month, product_id)
LEFT JOIN products p ON p.id = r.product_id
ORDER BY 1,2;Example 2 — Review a chart spec (JSON)
Open chart JSON
{
"mark": "line",
"encoding": {
"x": {"field": "date", "type": "temporal"},
"y": {"field": "revenue", "type": "quantitative"},
"color": {"field": "region", "type": "nominal"}
},
"data": {"name": "monthly_revenue"}
}- Accuracy: Add y-axis title with currency and formatting; confirm aggregation.
- Defaults: Specify timeUnit to monthly if upstream data is daily.
- Accessibility: Use color palette with sufficient contrast; add tooltip for exact values.
Improved spec
{
"mark": {"type": "line", "interpolate": "monotone"},
"encoding": {
"x": {"field": "date", "type": "temporal", "timeUnit": "yearmonth", "title": "Month"},
"y": {"field": "revenue", "type": "quantitative", "aggregate": "sum",
"title": "Revenue (USD)", "axis": {"format": "$~s"}},
"color": {"field": "region", "type": "nominal", "title": "Region"},
"tooltip": [
{"field": "date", "type": "temporal", "timeUnit": "yearmonth"},
{"field": "region", "type": "nominal"},
{"field": "revenue", "type": "quantitative", "aggregate": "sum", "format": ".2s"}
]
},
"data": {"name": "monthly_revenue"}
}Example 3 — Review a Python transform
Open code
def add_conversion_rate(df):
# df has clicks and purchases
df['conversion_rate'] = df['purchases'] / df['clicks']
return df- Correctness: Division by zero? Use safe denominator handling.
- Type safety: Ensure numeric dtypes; handle nulls.
- Documentation: Add docstring and example.
Improved code
def add_conversion_rate(df):
"""Add conversion_rate = purchases / clicks (0 when clicks == 0).
Expects numeric columns: clicks, purchases."""
clicks = df['clicks'].fillna(0)
purchases = df['purchases'].fillna(0)
df['conversion_rate'] = purchases.div(clicks.replace(0, float('inf'))).fillna(0.0)
return dfRecommended review process
Before you review
- Read PR title/description; run locally or open preview
- Skim files changed; identify high-risk areas first
- Open the checklist and keep it beside you
During review
- Comment on behavior and outcomes, not just syntax
- Be specific: point to lines, suggest concrete changes
- Batch nits; propose automated lint/pre-commit for repetitive items
After review
- Leave a summary comment: what you checked and remaining risks
- Approve with required changes or request updates
- Celebrate improvements; note follow-ups as tasks
Exercises (hands-on)
Exercise 1 — Review a SQL diff for a revenue metric
Instructions:
- Read the diff. Imagine it is a PR changing a KPI.
- Write 5–7 actionable review comments that reference lines and explain the impact.
- Suggest at least one test or validation step.
Open diff
diff --git a/metrics/revenue.sql b/metrics/revenue.sql
--- a/metrics/revenue.sql
+++ b/metrics/revenue.sql
@@
-SELECT DATE_TRUNC('month', order_date) AS month,
- SUM(amount) AS revenue
-FROM orders
-WHERE status = 'completed'
-GROUP BY 1
+SELECT DATE_TRUNC('month', order_date) AS month,
+ SUM(amount) AS revenue
+FROM orders o
+LEFT JOIN order_refunds r ON r.order_id = o.id
+WHERE o.status = 'completed'
+GROUP BY 1Tips
- Check correctness: refunds joined but never used?
- Timezones and net vs gross definition
- Performance and join type
Show solution
Example comments:
- (Lines +2,+3) We join refunds but never subtract refund amounts; this still reports gross revenue. Suggest SELECT SUM(o.amount) - COALESCE(SUM(r.refund_amount),0) AS net_revenue.
- (Line +1) Specify timezone in DATE_TRUNC to avoid cross-region drift.
- (Join) LEFT JOIN may duplicate orders if multiple refunds per order; aggregate refunds in a CTE before joining to prevent double-subtracting.
- (Filter) Ensure refunded orders still counted as completed when appropriate; clarify business rule in PR description.
- (Perf) Add predicate on r.refund_date to limit scanned partitions.
- (Validation) Add a check comparing net_revenue <= gross to catch sign issues.
- (Docs) Update metric definition to explicitly say “net of refunds.”
Exercise 2 — Improve a Pull Request description
Instructions:
- Rewrite the PR description below to a clear, complete one.
- Include: context, change summary, screenshots, risk/impact, test plan, rollout/owner.
Original PR description
Update revenue stuff and fix the chart. Also changed some names.Show solution
Sample improved description:
- Title: Net Revenue metric and monthly trend chart
- Context: Finance requested net revenue (gross - refunds) for monthly reporting.
- Changes: Updated metrics/revenue.sql to subtract refunds (see CTE refunds), adjusted chart spec to show monthly sum by region, added tooltip/axis labels.
- Screenshots: Attached before/after chart images with sample month.
- Risk/Impact: Slight metric decrease vs previous reports; dashboards depending on revenue will update automatically.
- Test plan: Queried last 6 months; verified net_revenue <= gross; compared two random orders manually.
- Rollout/Owner: Merge after approval; I will monitor dashboard refresh next morning.
Common mistakes and how to self-check
- Reviewing style before correctness: Start with metric logic and privacy.
- Vague comments: Always include what/why and a suggested fix.
- Overloaded PRs: Encourage small, focused changes.
- Unverified visuals: Ask for screenshots/GIFs with sample filters.
- Ignoring edge cases: Timezones, nulls, divisions by zero.
Self-check prompts
- Did I verify business logic matches the definition?
- If this broke tomorrow, would my comments help fix it fast?
- Could automation (lint, tests) prevent this class of issue next time?
Mini challenge (10 minutes)
Pick one dashboard you own. Create a 6-point review checklist specific to it (e.g., filters, data freshness, axis units, color mapping, default date range, accessibility). Share it with your team for future PRs.
Learning path
- Foundations: Git branches, commits, PRs
- Review mechanics: Checklists, respectful feedback, scope control
- BI specifics: Metric definitions, chart accuracy, data privacy
- Automation: Pre-commit hooks, SQL lint, visual regression screenshots
- Scaling: Review rotations, templates, and dashboards of dashboards
Practical projects
- Create a PR template for BI/viz changes (include test plan and screenshots)
- Build a SQL lint + unit test pipeline for a core metric
- Set up visual snapshot testing for a key chart spec
- Write a reviewer guide for your team with examples and do/don'ts
Next steps
- Adopt the checklist in your next 3 reviews
- Propose a PR template in your repo
- Automate one repetitive review item with tooling
About the Quick Test
Everyone can take the Quick Test for free; only logged-in users get saved progress.