Who this is for
Business Analysts, Product Analysts, and PM/BA hybrids who help teams choose what to build next and need a consistent, lightweight way to estimate Impact and Confidence for backlog items.
Prerequisites
- Basic understanding of user or business metrics (e.g., conversion, retention, NPS).
- Ability to read simple experiment or analytics summaries.
- Familiarity with backlog items (features, bugs, tech debt) and effort estimates from engineering.
Why this matters
In the job, you will be asked questions like: Which of these five features should we tackle this sprint? What’s the expected lift if we fix this issue? How sure are we? Impact and Confidence estimation helps you quickly and transparently compare items using the same yardstick, making prioritization faster and less subjective.
- Stakeholder planning: defend the roadmap with clear rationale.
- Sprint planning: pick high-value, high-confidence items early.
- Risk management: highlight low-confidence bets and propose evidence to de-risk.
Concept explained simply
Impact answers: If this succeeds, how big is the positive outcome? Confidence answers: How sure are we about that outcome?
You plug those into simple formulas teams already use:
- RICE Score = (Reach × Impact × Confidence) ÷ Effort
- ICE Score = (Impact × Confidence × Ease)
In this subskill, we focus on estimating the two trickiest parts: Impact and Confidence.
Simple, shared scoring rubrics you can adopt today
Impact (choose one):
- 0.25 = Minimal: tiny, hard-to-notice improvement
- 0.5 = Low: small lift for a small segment
- 1 = Medium: visible lift or reduces a common pain
- 2 = High: meaningful lift on a key metric or many users
- 3 = Massive: moves a north-star metric for a broad audience
Confidence (choose one):
- 0.5 = Low: opinion-based, sparse data
- 0.8 = Medium: some data or analogous cases
- 1.0 = High: strong evidence (e.g., experiment, reliable historical data)
Tip: Keep the rubric short so the team actually uses it. Consistency beats precision.
Mental model
Think of Impact as the height of the benefit and Confidence as the thickness of the evidence under it. A tall benefit on wobbly evidence is a risky skyscraper; a moderate benefit on solid evidence is a sturdy building. Put sturdy buildings in the critical path; explore skyscrapers when you have bandwidth or can cheaply de-risk.
How to estimate Impact and Confidence (repeatable steps)
- Define the outcome metric. What measurable change do we expect? (e.g., +X% conversion, -Y% time to task)
- Estimate Impact using the rubric. Translate expected change into the scale: Minimal → Massive.
- Audit evidence quality. What do we have: data, experiments, analogous cases, expert input?
- Assign Confidence from the rubric. Low (0.5), Medium (0.8), High (1.0).
- Note assumptions and next test. Record the biggest uncertainty and the fastest way to shrink it.
Evidence ladder (use to set Confidence)
- Expert opinion only → Confidence 0.5
- Light data (e.g., small sample user tests, directional analytics) → Confidence 0.6–0.7
- Analogous cases, historical wins in similar contexts → Confidence 0.7–0.8
- Strong correlational data (segmented, recent) → Confidence 0.8–0.9
- Well-powered experiment or robust causal evidence → Confidence 0.9–1.0
Pick the lowest tier that represents your weakest critical evidence.
Worked examples
Example 1 — Dashboard saved filters
- Outcome metric: weekly active saved-filter users
- Assumption: boosts analyst productivity, more return visits
- Impact: High → 2
- Confidence: Medium (analytics + 5 user interviews) → 0.8
- Reach: 2,000 users/month
- Effort: 2 person-months
RICE = (2,000 × 2 × 0.8) ÷ 2 = 1,600
ICE (assuming Ease=3) = 2 × 0.8 × 3 = 4.8
Example 2 — Fix duplicate invoices bug
- Outcome metric: refund rate reduction
- Impact: Massive → 3 (affects revenue and trust)
- Confidence: High (clear logs + support tickets) → 1.0
- Reach: 8% of transactions
- Effort: 0.5 person-month
RICE = (0.08 × 3 × 1.0) ÷ 0.5 = 0.48 ÷ 0.5 = 0.96
Note: Reach can be a fraction; teams often scale Reach to a common unit. If you scale monthly tx to 100,000, Reach=8,000 and RICE = (8,000 × 3 × 1) ÷ 0.5 = 48,000
ICE (Ease=4) = 3 × 1.0 × 4 = 12
Example 3 — Tooltip for a complex field
- Outcome metric: completion rate of a form
- Impact: Low → 0.5
- Confidence: Low (opinion, no data yet) → 0.5
- Reach: 4,000 users/month
- Effort: 0.2 person-month
RICE = (4,000 × 0.5 × 0.5) ÷ 0.2 = (1,000) ÷ 0.2 = 5,000
ICE (Ease=5) = 0.5 × 0.5 × 5 = 1.25
Insight: Low Impact can still win if Reach is large and Effort is tiny. That is why consistent definitions matter.
How to keep estimates consistent across items
- Use the same Impact/Confidence rubric every time.
- Write a one-liner for why you chose the score.
- When new evidence arrives, update Confidence only (don’t retro-fit Impact without reason).
Exercises (practice inside your backlog)
Do these now. They mirror the exercises below and take ~20–30 minutes.
- Compute ICE and RICE for 4 items. Apply the rubrics, write your reasoning in one line each, and rank items.
- Raise Confidence with fast evidence. Draft a 7-day plan to move one item from 0.5 to 0.8 Confidence.
Checklist before you prioritize
- Have I named the outcome metric?
- Did I choose Impact from the shared rubric (not a guessy number)?
- Did I set Confidence from the evidence ladder?
- Is the reasoning written in one clear sentence?
- Do I know the cheapest next test to increase Confidence?
Common mistakes and how to self-check
- Inflating Impact to win debates. Fix: use the rubric wording; compare to a known reference item.
- Confusing Confidence with priority. High Confidence does not mean high value. Check the actual Impact.
- Mixing units for Reach/Effort across items. Fix: define a common timeframe and effort unit.
- Not writing assumptions. Fix: add a 1–2 line note; it speeds up reviews and experiments.
- Never revisiting scores. Fix: update after new data or at a cadence (e.g., every sprint).
Self-audit mini-questions
- Which item has the lowest Confidence but high Impact? What’s the smallest test to learn more?
- Which high-Impact item has tiny Reach? Should it still be prioritized?
- If two items tie, what uncertainty breaks the tie?
Practical projects
- Backlog calibration session: Pick 8–12 items, estimate Impact/Confidence in 30 minutes, and publish a one-page rationale.
- Evidence sprint: For 2 risky bets, run 1 usability test + 1 data pull each to move Confidence from 0.5 → 0.8.
- Rubric rollout: Create a one-pager with your Impact and Confidence rubrics and 3 examples; share with your team.
Learning path
- Before this: Metrics basics, writing clear problem statements.
- Now: Impact and Confidence estimation with the provided rubric.
- Next: Turn estimates into prioritized lists with RICE/ICE, and design quick experiments to de-risk low-confidence items.
Next steps
- Apply the rubrics to your current sprint candidates.
- Add a short "assumptions and test" note under each item.
- Schedule a 15-minute review with your team to align and adjust.
Note: The Quick Test is available to everyone; only logged-in users get saved progress.
Mini challenge
Pick one high-Impact, low-Confidence item in your backlog. In 5 sentences max, write: the outcome metric, Impact score + reason, Confidence score + reason, and the smallest test to increase Confidence within 3 days.