Analytics and measurement
A/B Test Sample Size Calculator
Work out how many visitors an A/B test needs before you start, and roughly how many days that takes on your traffic.
Runs in your browser; nothing is uploaded. · Free, no account · 2 min
Importing a saved run into your account is coming soon. Until then, your last runs stay in this browser only, and you can download your files.
How to use
- Enter your current conversion rate.
- Enter the smallest change worth detecting, as a relative lift or in percentage points.
- Choose the significance level and the power.
- Add your daily visitors to see how long the test would run.
What it checks and how it works
The tool uses the normal-approximation formula for comparing two proportions with an equal split, as documented in the NCSS PASS chapter on tests for two proportions: the sample per version grows with the spread of the two rates and with the square of the z-values for the significance level and the power, and shrinks with the square of the gap between the rates. Halving the change you want to detect roughly quadruples the visitors you need.
It is a planning figure. It assumes two versions, one primary metric and a decision made once the planned sample is reached. More variants or metrics need a larger sample or a correction for multiple comparisons. Defaults of 5% significance and 80% power are conventions, not rules; choose values that match the cost of a wrong call.
Sources
Related tools
A/B Test Significance Calculator
Check whether the gap between two versions is bigger than chance would explain, and how big the difference might really be.
Conversion Rate Calculator with Confidence Range
Work out a conversion rate and the range it could plausibly sit in, given how much traffic stands behind it.
AI Answer Sample-Size Calculator
Find how many AI answers you need to measure a mention rate within a margin of error, or the margin your answers give.
ROAS Calculator with Break-Even
Work out return on ad spend, and whether it covers your costs once you add your gross margin.
Go further with Zaher: See AI-assistant traffic next to your other channels
Questions
About this tool
Power is the chance that the test detects a difference of the size you specified, if it really exists. At 80%, one test in five of that size of effect would still miss it.
The sample needed grows with the square of how small the gap is. Detecting a 1% lift takes about a hundred times the visitors of detecting a 10% lift.
Test a bigger change, test a page or step with more traffic, or measure a metric earlier in the funnel. A test that cannot reach its sample size is not worth starting.
Get started today
Find out where your brand stands in AI. Free.
Run a free AI visibility analysis on your domain. No credit card, results in minutes. See what AI engines say about your brand today.
Brands already measuring their AI visibility with Zaher.
