Clixcel Free tools

Free tool · No sign-up · Nothing stored

A/B test calculator

Check whether your test result is real or just noise, and work out how many visitors a test needs before you start it.

Used to spot a sample ratio mismatch.

How the result is worked out

The calculator runs a two-sided test on the two conversion rates, the standard method for comparing proportions. The range shown for the improvement is where the true change most likely sits at your chosen confidence level. When the range crosses zero, the test has not shown a difference yet.

Sample sizes use the same method in reverse. With more than two versions, the confidence level is tightened for each comparison so the chance of a false win stays where you set it.

Questions

What does statistically significant mean?

That the difference between the versions is unlikely to be down to chance alone. At 95% confidence, a difference this large would appear less than 5% of the time if the two versions really performed the same. It does not tell you the size of the gain is certain, which is why the range matters too.

Can I stop the test as soon as it turns significant?

No. Checking every day and stopping at the first good result inflates false wins badly. Work out the sample size before you start, run the test until you reach it, and run for whole weeks so weekdays and weekends are both counted.

What is a sample ratio mismatch?

When traffic was meant to be split evenly but one version received noticeably more visitors than chance allows. It usually means a tracking or redirect problem, and results from such a test should not be trusted until the cause is found.

What is minimum detectable effect?

The smallest improvement you want the test to be able to find. Smaller effects need far more visitors: halving it roughly quadruples the sample size. Pick the smallest change that would be worth acting on.

Need tests that actually move revenue?

Our conversion team researches where visitors drop off, designs the tests, and reads the results with the care they need.

See our CRO service