Note: The quick test is available to everyone. Only logged-in learners will have their progress saved.
Why this matters
As a Marketing Analyst, you watch metrics like spend, conversions, CPA, CTR, and ROAS. Spikes and drops can signal real business events: broken tracking, paused campaigns, creative fatigue, site outages, or sudden viral traffic. Well-designed alerts catch issues early without overwhelming you with noise.
- Daily operations: Get notified when spend surges unexpectedly or conversions crash.
- Campaign health: Detect CTR drops after a creative change.
- Attribution sanity checks: Catch tracking breaks when events suddenly vanish.
- Budget control: Flag ROAS dips to prevent wasted spend.
Concept explained simply
An alert is a rule that compares the latest value to a baseline and decides if the difference is unusual enough to notify you. You choose the metric, a baseline (e.g., last 7-day average or same weekday last week), a threshold (e.g., 30% drop), volume guardrails, and a cooldown to prevent repeat noise.
Mental model
Think of alerts like a smoke alarm with three knobs:
- Sensor: Which metric to monitor (e.g., conversions, CPC, CTR).
- Sensitivity: How big and how fast a change must be to matter.
- Context: What "normal" looks like (time of day, day of week, seasonality).
Common baselines and when to use them
- Last value vs. last 7-day mean: Good for stable metrics.
- Week-over-week same weekday: Reduces day-of-week effects.
- Rolling median and MAD: Robust to outliers in noisy data.
- Holidays/event windows: Use custom baseline or temporarily disable.
Detection methods — quick guide
- Absolute threshold: Alert if Spend today > 10,000 (budget guardrail).
- Relative change: Alert if Conversions drop by 30% vs 7-day average.
- StdDev/MAD: Alert if metric deviates by more than 3σ or 3×MAD from baseline.
- Rate metrics with volume guardrails: Only alert on CTR changes if impressions ≥ 5,000.
- Seasonality-aware: Compare to same weekday or same hour baseline.
Alert hygiene: reduce noise
- Cooldown window (e.g., 6–24h) to avoid repeated alerts for the same issue.
- Minimum data volume to prevent false signals on tiny samples.
- Severity levels (e.g., minor: 15–30%, major: 30–60%, critical: 60%+).
- Routing: Send critical alerts to broader channels; minor to a daily digest.
- Quiet hours: Defer non-critical alerts overnight.
Worked examples
-
CTR drop with volume guardrail
Yesterday CTR: 2.5%. 7-day average CTR (same weekday): 3.2%. Impressions today: 40,000. Change: (2.5 - 3.2) / 3.2 = -21.9%.
Rule: Alert on CTR drop > 20% if impressions ≥ 10,000. Result: Trigger alert (major).
-
Spend spike with cooldown
Baseline daily spend: 8,000 (7-day mean). Today at 11:00, spend-to-date: 6,000 vs typical 3,000 by 11:00.
Intraday rule: Alert if cumulative spend by hour exceeds typical by 80% and day total projected > 12,000. Result: Trigger once, cooldown 12h.
-
Conversions anomaly with MAD
Last 14 same-weekday conversions: median = 420, MAD = 35. Today: 300.
Robust z-score ≈ (300 - 420) / (1.4826 × 35) ≈ -2.3. Rule threshold: |z| ≥ 3. Result: No alert (watchlist only).
How to design useful alerts (step-by-step)
- Define the decision: What action will you take if this fires? If none, don’t alert.
- Pick metric and slice: e.g., ROAS by channel, conversions by campaign, CPC by device.
- Choose baseline: 7-day average, WoW same weekday, or robust median.
- Set thresholds: Start with 30% change; tighten/loosen after a week of observation.
- Add guardrails: Minimum volume, cooldown, and severity levels.
- Test on history: Backtest 30–90 days to estimate false positives/negatives.
- Roll out: Start as digest; promote to real-time channel after tuning.
Backtesting mini-steps
- Pick a time window with known incidents (tracking break, budget change).
- Simulate alerts daily/hourly and count matches with known incidents.
- Adjust thresholds until you capture key incidents with tolerable noise.
Exercises
Do these before the quick test. Mirror of the Exercises section below.
- Exercise 1: Create an alert plan for CTR and ROAS with baselines and thresholds.
- Exercise 2: Compute alert triggers from sample data and decide severity.
Pre-flight checklist
- I selected a baseline that matches the metric’s seasonality.
- I set a minimum volume to avoid small-sample noise.
- I defined cooldown and severity levels.
- I wrote the concrete action to take when the alert fires.
Common mistakes and self-check
- No volume guardrail: Alerts on 12 impressions. Fix: require minimum impressions/clicks/conversions.
- Wrong baseline: Comparing weekends to weekday averages. Fix: use same weekday baseline.
- Too many alerts: Alert fatigue. Fix: cooldowns, severity, daily digests for minor signals.
- Ignoring data latency: Triggering on partial data. Fix: intraday thresholds by hour; wait windows.
- Ambiguous ownership: No clear action. Fix: add runbook in alert message (what, who, when).
Self-check prompt
If this alert fired at 2am, what exact action would I take within 15 minutes? If you can’t answer, refine the rule.
Practical projects
- Build a channel health alert pack: spend spike, conversions drop, ROAS dip, CTR drop, CPC surge.
- Create a weekly noise report: counts of alerts by severity, false-positive rate, and tuning suggestions.
- Implement a campaign onboarding checklist: default alert templates applied to each new campaign with baseline preview.
Who this is for
- Marketing Analysts maintaining BI dashboards and growth reports.
- Performance marketers who own budgets and channel health.
- Analysts migrating manual checks to automated monitoring.
Prerequisites
- Comfort with core metrics (impressions, clicks, CTR, CPC, conversions, CPA, ROAS).
- Basic statistics: averages, medians, standard deviation, percentage change.
- Familiarity with your BI tool’s scheduling and alerting features.
Learning path
- Review metric definitions and trustworthy data sources.
- Practice baselines: 7-day mean vs same weekday vs rolling median.
- Add guardrails: minimum volume, cooldown, severity.
- Backtest on past incidents and tune thresholds.
- Roll out to real-time channels with a runbook for actions.
Mini challenge
Design a single cross-channel "Critical ROAS Dip" alert that triggers only when:
- ROAS drops by at least 35% vs same weekday baseline,
- and Spend today ≥ 3,000,
- and at least 2 channels show the dip simultaneously.
What to hand in
- The exact rule logic and baseline definition.
- Severity classification and cooldown.
- The runbook steps the on-call analyst should follow.
Next steps
- Implement one high-impact alert this week (start with conversions drop).
- Backtest and adjust threshold after 7 days.
- Add ownership and a short runbook to every alert message.