A/B Test Sample Size Calculator

How to use it

1. Enter your current conversion rate (the control, in %).
2. Choose the minimum detectable effect: the smallest lift that would still change your decision, relative (%) or absolute (points).
3. Keep 95% confidence and 80% power (the market defaults) or adjust if you know why.
4. Enter your total weekly traffic to see the estimated duration.
5. Read the result: visitors per variation, total for two variations and days to finish.

How it works: the formula

The calculator uses the two-proportion normal approximation, the same method behind most serious A/B testing tools. Sample size per variation is:

Where p₁ is the baseline rate, p₂ is the target rate (baseline plus the MDE), p̄ is their average, z_α is the confidence critical value (1.96 for 95% two-sided) and z_β is the power one (0.84 for 80%).

Worked example (reproduces the default result)

With the values that come pre-filled: baseline 5%, MDE +10% relative (target of 5.5%), 95% confidence and 80% power, two-sided. The absolute difference is 0.5 point (0.055 − 0.05). Plugging into the formula, the numerator is about 0.7808 and the denominator 0.000025, so:

n ≈ 0.7808 / 0.000025 = 31,234 visitors per variation, or 62,468 total for two versions. At 10,000 visitors per week (about 1,429 per day), reaching 62,468 takes 44 days. That is exactly what the tool shows above when you open the page.

How to read it, and where it fools you

The per-variation number is the target you fix before you start and do not change midway. Rule of thumb: the smaller the effect you want to detect, the much larger the sample. So be honest about the MDE, promising to detect +2% relative costs a sample most sites will not get in months.

Limits to keep in mind: the math assumes stable traffic and a binary metric (converted or not). It does not cover revenue per visitor (higher variance, needs more sample), strong seasonality, or many variations at once. If your baseline is very low (under 1%) or your traffic is small, first check whether a test is feasible with the significance calculator and the guide on how to run an A/B test.

Best practices when sizing

Before you turn the test on, lock three things: the sample per variation, the minimum effect you accept and the runtime. After that, follow these simple rules.

• Prefer the smallest MDE that would still change your decision. An inflated MDE hides real gains and makes you stop too early.
• Run whole weekly cycles to capture the difference in behavior between weekdays and weekends.
• One primary metric per test. Chasing several at once multiplies the false positive risk.
• If the required sample is bigger than a full month of your traffic, raise the target effect or gather more traffic before you start.

Frequently asked questions

How many visitors does an A/B test need?

It depends on three things: your current conversion rate, the smallest effect you want to detect and your confidence/power levels. The lower the baseline rate and the smaller the target effect, the larger the sample. For a 5% rate detecting a +10% relative lift, at 95% confidence and 80% power, that is about 31,000 visitors per variation.

What is the minimum detectable effect (MDE)?

It is the smallest lift worth detecting, expressed as relative (+10%) or absolute (+2 points). It drives sample size more than anything else: detecting +2% relative costs a huge sample, while detecting +20% is far cheaper. Pick the smallest effect that would still change your decision.

Why 95% confidence and 80% power?

They are the market conventions. 95% confidence means accepting a 5% false positive rate. 80% power means an 80% chance of detecting the effect if it is real. Raising either one increases the sample you need.

Can I stop the test as soon as I hit the sample?

Yes, if you fixed the sample and the runtime before you started. The common mistake is the opposite: peeking at the result and stopping the first time it looks significant, which inflates the false positive rate. Set the sample here, run until it is complete, then read the verdict.

Does it work for tests with more than two variations?

The per-variation number still holds, but the total runtime grows because traffic splits across more groups. To plan duration with 3 or more variations, use the A/B test duration calculator.

Keep going

Sized the test? Run until the sample is complete, then check the result with the statistical significance calculator. To plan the calendar with more than two variations, use the A/B test duration calculator. The full context is in the guide on what A/B testing is.