Experiment Calculator

Baseline Conversion Rate (%) *

Enter as a number (e.g., 2.5 for 2.5%).

Desired Lift (%) *

Relative improvement you want to detect (e.g., 10).

Statistical Power (%)

Probability of detecting a true effect (80% is typical).

Confidence Level (%)

Scenario Mode

Results

5,610

visitors per variant

Total sample: 11,220 visitors (Control + Variant)

New conversion rate target: 2.75% (2.5% + 10% lift)

Assumptions used:

Baseline: 2.5%, Lift: 10%, Power: 80%, Confidence: 95% (two‑tailed)

Understanding Experiment Lift & Sample Size

Why this matters

Properly sizing an A/B test prevents wasted traffic and time. An under‑powered test may miss a real improvement, while an oversized test delays learning. This calculator helps you plan the traffic needed to detect a meaningful change with a chosen level of confidence.

What each input means

Input	What it represents	Where to find it	Common pitfall
Baseline Conversion Rate	Current performance of your control (e.g., existing page, current email).	GA4, your CRM, or ad platform conversion reports.	Using a short‑term spike; use a stable period (e.g., last 30‑90 days).
Desired Lift	Minimum improvement you want the test to detect reliably.	Business goal or historical test lifts.	Choosing too small a lift (requires huge traffic) or too large (may miss meaningful changes).
Statistical Power	The probability that the test will detect a true effect of the size you specify.	Set by your risk tolerance; 80% is standard.	Lower power increases the chance of a false negative.
Confidence Level	The probability that the observed difference is real and not due to random chance.	Standard is 95% (5% significance level).	Confusing confidence with the probability that your variant is better.

Step‑by‑step guide for this tool

Enter your current conversion rate (as a percentage, e.g., 2.5).
Enter the relative lift you aim to detect (e.g., 10 for a 10% improvement).
Adjust power and confidence if your experiment requires higher or lower certainty.
Review the required visitors per variant and total sample size.
Use the “What‑If” helper below to see how changes in lift or baseline affect sample needs.

Realistic examples

Example 1: E‑commerce checkout test

Baseline conversion rate: 3.2% (from analytics). Desired lift: 15% (to 3.68%). Power: 80%, Confidence: 95%. Result: 3,456 visitors per variant needed. At 500 visitors/day per variant, the test would need about 7 days to reach significance.

Example 2: Landing page sign‑up test

Baseline: 8% (high‑intent traffic). Desired lift: 5% (to 8.4%). Power: 90%, Confidence: 99%. Result: 16,224 visitors per variant needed. This illustrates how higher confidence and power dramatically increase required sample size.

Interpretation guide

If your required sample is huge: Your desired lift may be too small, or your baseline too low. Consider whether the business impact justifies the traffic/time.
If your required sample is small: The expected lift is large or baseline high. Ensure the lift is realistic based on past tests.
Double‑check: Are you using a consistent time period for baseline? Is your traffic evenly split between control and variant?

Common mistakes checklist

Stopping the test early because results “look” good Not accounting for seasonality in baseline data Changing the test goal/metric mid‑experiment Ignoring unequal traffic split between variants

What‑If: Lift vs. Sample Size

See how the required sample changes with different lift or baseline assumptions.

Baseline Conversion Rate (%): 2.5

Desired Lift (%): 10

5,610

visitors per variant needed

Frequently Asked Questions

How accurate is this calculator?

It uses standard statistical formulas for two‑proportion Z‑tests. Accuracy depends entirely on your inputs. Real‑world factors like day‑of‑week effects, traffic quality shifts, or implementation errors can affect actual results.

Where do I get a reliable baseline conversion rate?

Use your analytics platform (GA4, Adobe Analytics) over a stable, recent period—typically 30–90 days. Exclude outliers like holidays or campaign spikes. Ensure you’re measuring the exact same conversion action you plan to test.

What if I have more than two variants (A/B/n)?

This calculator shows per‑variant sample. For n variants, multiply the per‑variant result by n. However, for multiple comparisons, you may need to adjust significance levels (e.g., Bonferroni correction), which increases sample size.

Should I run the test until I hit the exact sample size?

Ideally, yes. Stopping early because results look favorable increases the risk of false positives. Use a predetermined sample size and duration, and analyze only at the end (or use sequential testing methods).

Lift Calculator for Experiments

Menu

Lift Calculator for Experiments

AI Assistant

Was this tool helpful?