Is your test result actually significant?
Variant A (Control)
Conversion rate
5.00%
Variant B (Challenger)
Conversion rate
7.00%
Result
⏳ Not significant yet
Confidence
94.0%
Uplift
+40.0%
Z-score
1.88
p-value
0.0597
You need p < 0.05 (confidence > 95%) before calling a winner. Keep running the test.
Sample size calculator
Needed per variant
8,149 visitors
Estimated test duration
82 days
Statistical significance
A result is significant when there's less than a 5% chance it's due to random chance (p < 0.05, confidence > 95%). Below this, you're flipping a coin.
Minimum detectable effect
The smallest improvement you care about detecting. If you want to detect a 10% uplift, you need a much larger sample than detecting a 50% uplift.
Don't stop early
Checking results daily and stopping when you see a winner inflates your false positive rate to 26%+. Run tests until you hit the required sample size.
One change at a time
Only change one element per A/B test. Testing multiple things simultaneously makes it impossible to know what caused the change. For multiple changes, use multivariate testing.
You've got the numbers — now ship the features that move them. We help founders scale their app or SaaS with new features, performance, and growth experiments. No bloated retainers, just fast execution.