Who this is for
Applied Scientists and ML practitioners who need to turn broad business ideas into a deliverable ML scope and decide if a project is worth doing now, later, or not at all.
Prerequisites
- Basic ML knowledge (supervised learning concepts, evaluation metrics)
- Comfort with exploratory data analysis
- Understanding of product metrics and simple ROI math
Why this matters
Great models fail when the scope is wrong or feasibility is overlooked. In real teams, you will:
- Decide if a problem needs ML or a simpler heuristic
- Estimate effort, data needs, and integration cost
- Align stakeholders on what is in/out of v1
- Protect timelines by flagging risks early
Real tasks you’ll face
- Turning “reduce churn” into a specific decision, metric, and target user segment
- Checking if labels exist or can be collected without violating policy
- Choosing between batch scoring vs. real-time to meet latency constraints
- Preventing scope creep by setting clear out-of-scope items
Concept explained simply
Scope = what you will deliver by when, to whom, and what is explicitly out-of-scope. Feasibility = can you do it with the data, time, team, tools, and constraints you have?
Answer four questions:
- Decision: What decision or action will the model influence?
- Value: Why does this matter now, and how will we measure success?
- Resources: Do we have enough data, people, and compute to ship v1?
- Constraints: What legal, ethical, latency, or integration limits apply?
Mental model: The 7-Box Feasibility Check
- Problem clarity: Decision, users, target segment
- Success metrics: Primary (e.g., AUC, lift, revenue), guardrails (latency, fairness)
- Data: Availability, quality, labeling cost, leakage risks
- Technical: Baseline, model options, infra, latency/throughput
- Operations: Integration points, monitoring, retraining
- Business: Expected impact vs. cost, risks, compliance
- Plan: Milestones, out-of-scope list, go/no-go gates
Step-by-step: Scope and feasibility in 7 steps
- Write a one-sentence scope
Template: “We will deliver [artifact] that helps [user] decide [decision] for [segment] by [date], measured by [metric]. Out-of-scope: [list].” - Define success and guardrails
- Primary: measurable outcome (e.g., +3% conversion)
- Technical: e.g., AUC≥0.78 vs baseline 0.72
- Operational: e.g., p95 latency ≤ 80ms
- Ethical: fairness gap ≤ 3pp across groups
- Data triage
- Target rate, sample size, missingness
- Label availability and cost
- Time leakage checks (train/test split by time)
- Baseline first
- Heuristic or simple model as a yardstick
- Define “ship if” rule: must beat baseline by X
- Tech and infra constraints
- Latency and throughput requirements
- Batch vs. real-time scoring
- Compute/storage budget
- Business case (quick math)
- Value ≈ Impacted volume × Improvement × Value per unit − Cost
- Document risks and mitigations
- Stage the delivery
- M0: data audit and baseline
- M1: prototype and A/B readiness
- M2: launch and monitoring
Mini tasks for each step
- Rewrite the scope to remove ambiguity
- Choose 1 primary metric and 2 guardrails
- List top 3 data risks and a mitigation each
- Sketch a zero-ML baseline (rule or lookup)
- Decide batch vs. real-time and justify
- Compute a back-of-envelope ROI
- Define a go/no-go gate after M0
Worked examples (3)
Example 1: Churn prediction for subscription app
- Decision: Offer retention incentive to likely churners
- Success: +2 pp retention at Day 30; guardrails: incentive cost per saved user ≤ $2
- Data: Labels exist (cancel events); features: usage logs; leakage risk from post-cancel interactions
- Baseline: Target top 5% least active users by past week; historical uplift +0.8 pp
- Tech: Batch daily is enough; no real-time needed
- Business: 1M users × 2 pp × $4 LTV ≈ $800k gross benefit vs $120k cost ⇒ feasible
- Scope: v1 excludes cross-device identity resolution
Example 2: Demand forecasting per SKU per store
- Decision: Set weekly order quantities
- Success: Reduce stockouts by 10% with ≤ 5% overstock
- Data: Sales history sparse for tail SKUs; seasonality present
- Baseline: Moving average with seasonal factor
- Tech: Batch weekly; latency not critical
- Risk: Cold-start for new SKUs; mitigation: hierarchical model + category priors
- Scope: v1 at category level, not SKU-store granularity
Example 3: Content moderation
- Decision: Queue content for human review if unsafe
- Success: 90% recall on unsafe with ≤ 5% false positives
- Data: Labels costly; long-tail rare harms
- Baseline: Keyword rules + hash matching
- Tech: Real-time ≤ 50ms at p95 ⇒ consider lightweight model
- Ethics: Bias checks across languages
- Scope: v1 English only; human-in-the-loop review
Hands-on exercises
These exercises are available to everyone. If you are logged in, your progress will be saved.
Exercise 1: Write a scope and feasibility checklist
Scenario: You’re asked to predict ad click-through rate (CTR) to improve bidding. The ads team wants real-time predictions.
- Write a one-sentence scope
- Choose 1 primary metric and 2 guardrails
- List the top 5 feasibility checks across data, tech, and business
Hints
- Latency: what is acceptable at p95?
- Baseline: current heuristic and its performance
Exercise 2: Stage the delivery and define go/no-go
Scenario: Predict product returns to trigger extra QC before shipping.
- Create a 3-milestone plan (M0–M2)
- Define a go/no-go gate after M0 with concrete criteria
- Mark at least 3 items out-of-scope for v1
Hints
- Think about label delay and leakage
- QC cost must not exceed avoided return cost
Self-check checklist
- Your scope identifies decision, user, segment, timeline, and out-of-scope
- Metrics include at least one outcome metric and operational guardrails
- Data risks include label availability and leakage checks
- You defined a baseline and a “ship if” rule
- You staged delivery with a clear go/no-go gate
Common mistakes and how to self-check
- Vague scope: If two people can interpret it differently, rewrite it. Include out-of-scope.
- Metric mismatch: Offline AUC ≠ business success. Tie to user or revenue impact.
- Ignoring latency: Choose batch if real-time gives little incremental value.
- Data leakage: Check that no post-decision features are in training.
- No baseline: Always compare to a simple rule/heuristic.
- Unbounded v1: Put a time-box and clear milestone gates.
Practical projects
- Baseline vs. model: Build a heuristic and a simple ML model for a public dataset; document a ship/no-ship rule
- Data audit: Create a data quality report (missingness, target rate, leakage tests)
- Scope doc + risk register: 1-page scope, metrics, and top risks with mitigations
Learning path
- Practice writing one-sentence scopes for 3 different scenarios
- Run a data feasibility checklist on a dataset (labels, target rate, leakage)
- Build and evaluate a baseline vs. simple model; define a “ship if” rule
- Plan a staged rollout with monitoring metrics
Next steps
- Complete the exercises above
- Take the Quick Test to validate understanding
- Apply the 7-Box checklist to your current project at work
Mini challenge
Pick any feature in a product you use daily. Define a one-sentence scope for an ML improvement, list three feasibility risks, and state a go/no-go criterion you would use.