Why this matters
Great data science starts with a clear, shared definition of the problem. Framing with stakeholders prevents wasted effort, aligns success metrics with business value, and reduces rework. Real tasks you will face:
- Translating a vague request ("build an AI model") into a decision-ready question tied to outcomes.
- Choosing the right level of solution (dashboard, rule, experiment, model) based on constraints.
- Defining success metrics, baselines, and guardrails that business leaders trust.
- Documenting assumptions, risks, and a timeboxed plan everyone signs off on.
Concept explained simply
Problem framing is a structured conversation that turns a business ask into a precise question with success criteria, constraints, and a plan. It answers: what decision will be made, by whom, when, using what evidence, and how we will know it worked.
A quick mental model
- Destination: The business outcome (e.g., reduce churn by 10%).
- Current location: Baseline performance and context (today's churn, process, data).
- Road rules: Constraints and guardrails (timeline, budget, compliance, customer impact).
- Route options: Levels of solution (query, report, heuristic, experiment, ML model).
- Milestones: Timeboxed plan with checkpoints and decisions.
Quick glossary
- Decision owner: Person accountable for acting on results.
- Primary metric: The KPI we intend to move.
- Guardrail metric: A KPI we must not harm while optimizing the primary metric.
- Baseline: Current value of the metric, used for comparison.
- Timebox: Agreed limit for exploration before a go/no-go decision.
Who this is for
- Data Scientists and ML Engineers who collaborate with product, marketing, ops, or finance.
- Analysts transitioning to DS who need to drive stakeholder conversations.
- New team leads who must align scope, metrics, and delivery trade-offs.
Prerequisites
- Basic understanding of KPIs and experimental design (AB tests or offline evaluation).
- Ability to read data schemas and identify available signals.
- Comfort presenting concise written summaries (one-pagers).
Step-by-step framing method
- Clarify the business goal
- Ask: What outcome are we trying to change, and why now?
- Capture: Decision owner, target audience, expected impact window.
- Define the decision and action
- Ask: What decision will this analysis/model inform? Who will act on it?
- Capture: Decision cadence (real-time, daily, quarterly) and action mechanism.
- Choose success and guardrail metrics
- Ask: How will we measure success? What must not get worse?
- Capture: Metric formula, unit of analysis, baseline, target, evaluation window.
- Map constraints and risks
- Ask: Timeline, budget, compliance, data access, operational limitations.
- Capture: Hard constraints (must-haves) vs soft preferences (nice-to-haves).
- Assess data feasibility
- Ask: Do we have target labels, volume, quality, and coverage? Any leakage risks?
- Capture: Data sources, availability, sampling plan, known gaps.
- Pick the smallest useful solution
- Decide: Start with a rule or report if it answers the decision fast; scale to ML if needed.
- Capture: Why this level of solution matches constraints and value.
- Timebox and align on milestones
- Propose: Milestone 1 (data check), Milestone 2 (baseline & proxy), Milestone 3 (experiment/rollout).
- Capture: Exit criteria for each milestone and go/no-go decision points.
- Document a one-page brief and confirm
- Share: Problem statement, metrics, constraints, plan, owners, and open questions.
- Confirm: Stakeholders agree in writing before build.
Reusable one-page framing template
Copy these headings into your doc and fill them:
- Goal: [business outcome] by [X%] in [Y weeks], because [why now]. Baseline: [value].
- Decision & Action: [who] will [do what] [when/how often] based on [artifact/output].
- Primary Metric: [name, formula, unit, window]. Target: [value].
- Guardrails: [metric] must stay ≥/≤ [threshold].
- Constraints: [timeline, budget, compliance, ops].
- Data: Sources [A/B/C], coverage [%], label availability, known gaps/leakage risks.
- Solution Scope: [query/report/rule/experiment/model], why smallest useful.
- Milestones & Exit Criteria: [M1], [M2], [M3].
- Owners & Stakeholders: Decision owner, DS owner, reviewers.
- Risks & Mitigations: [risk → mitigation].
Worked examples
Example 1: Reduce churn in a subscription app
- Situation: "We need a churn prediction model."
- Key questions asked: What decision will use this? What action follows a high-risk score? Baseline churn? Target window?
- Reframed problem: Reduce voluntary monthly churn from 6.5% to 5.5% within 12 weeks by prioritizing save-offers to high-risk users.
- Decision & action: Retention team will send a tailored offer within 24 hours of a high-risk flag.
- Primary metric: Monthly churn rate (customers canceling/current customers). Target: -1.0 pp vs baseline. Window: 12 weeks.
- Guardrails: Offer cost per save ≤ $4; NPS no worse than baseline.
- Constraint: Ops can handle 5k offers/day → cap predictions accordingly.
- Smallest useful: Start with a rules-based score from engagement features; evaluate uplift vs control; graduate to ML if needed.
Example 2: Improve delivery ETA accuracy
- Situation: "ETA model is wrong; make it better."
- Reframed problem: Reduce absolute ETA error from median 12 min to ≤ 7 min for urban deliveries this quarter.
- Decision & action: Routing system selects buffer time based on predicted uncertainty band; ops monitors SLA breaches.
- Primary metric: Median absolute error (minutes). Guardrail: On-time delivery rate ≥ 95%.
- Constraints: Real-time predictions ≤ 50 ms p95; limited GPS accuracy in some areas.
- Smallest useful: Add per-zone residual bias correction and uncertainty quantiles before full model retrain.
Example 3: Reduce false fraud alerts
- Situation: "Fraud team is overwhelmed; too many false positives."
- Reframed problem: Reduce manual review queue volume by 30% without increasing fraud loss per $1k transactions.
- Decision & action: Auto-approve low-risk transactions; escalate medium; block high-risk.
- Primary metric: Reviews per 1k transactions. Guardrail: Fraud loss/$1k ≤ baseline.
- Constraints: Regulatory explanations required for blocked cases; must produce reason codes.
- Smallest useful: Calibrate scores and adjust thresholds; add reason-code mapping for explainability.
Exercises
Complete the two exercises below. You can compare your work with the provided solutions. Tip: use the one-page template.
Exercise 1 — Frame a vague request into a decision-ready problem
Stakeholder message: "Marketing wants an AI to boost conversions fast. We have 3 weeks before the campaign. Can you build something?" Produce a concise problem statement including: goal, decision & action, primary metric (with formula), baseline, target, guardrails, constraints, data feasibility, smallest useful solution, milestones.
Show a possible structure
- Goal
- Decision & Action
- Primary Metric
- Baseline & Target
- Guardrails
- Constraints
- Data
- Solution Scope
- Milestones
Exercise 2 — Stakeholder map and discovery questions
Scenario: Product asks for "better recommendations" on the home page. Identify primary vs secondary stakeholders, then list six discovery questions and three hard constraints to confirm.
Hints
- Who owns the decision and impact?
- What works today and what breaks if traffic doubles?
- What must not get worse while optimizing engagement?
Self-check checklist
- Your problem statement names the decision owner and action mechanism.
- Metrics include formula, baseline, target, and evaluation window.
- Constraints are explicit and testable (time, budget, ops, compliance).
- Data feasibility covers labels, coverage, and leakage risks.
- Scope starts with the smallest useful solution and a timeboxed plan.
Common mistakes and how to self-check
- Mistake: Jumping to models before defining the decision. Self-check: Can you write "Who will do what, when" in one sentence?
- Mistake: Vague metrics ("increase engagement"). Self-check: Do you have a formula, unit, and time window?
- Mistake: Ignoring guardrails. Self-check: What harm could a naive optimization cause, and how will you detect it?
- Mistake: Over-scoping. Self-check: What is the smallest useful artifact that answers the decision?
- Mistake: Hidden constraints. Self-check: Have you listed compliance, ops capacity, and SLA limits?
Practical projects
- Write a one-page framing brief for a churn reduction initiative using a dataset you know. Present to a friend and refine based on their questions.
- Run a 30-minute mock stakeholder interview. Record the questions you asked and rewrite the problem twice: a minimal and a stretch version.
- Build a metric dashboard prototype for your primary and guardrail metrics with baselines and alert thresholds.
Learning path
- Start: Problem framing (this page) to align goals, metrics, and constraints.
- Next: Experiment design and evaluation to validate impact.
- Then: Communication of results and trade-offs to drive decisions.
- Finally: Productionization planning and monitoring with guardrails.
Mini challenge
Pick one product in your life (music app, food delivery, ride-share). Write a 6-sentence framing brief to improve a key outcome (e.g., "repeat orders"). Include decision owner, decision/action, primary metric with baseline/target, one guardrail, a hard constraint, and a 2-milestone plan.
Next steps
- Use the template on your next task; share with the decision owner for confirmation.
- Translate metrics into a small dashboard or query so everyone can see the baseline.
- Timebox your first milestone and schedule the go/no-go review now.
Quick Test and progress
The Quick Test below is available to everyone. If you are logged in, your progress will be saved automatically.