Sample-size planner
Pick a metric family — conversion rate or continuous — set the lift you want to detect, alpha, and power. AB SHARK returns the per-arm sample size.
Plan for proportions or continuous metrics
For conversion-rate tests, AB SHARK uses a two-proportion z formula where variance is determined by the baseline. For continuous metrics (means, revenue per user, time on site), it uses a Welch-based two-sample formula that takes the baseline standard deviation and the minimum detectable effect (MDE) in absolute or relative units.
Frequently asked questions
How do I calculate sample size for a continuous-metric A/B test?
Provide the baseline standard deviation, the minimum detectable effect (absolute or relative), alpha, and desired power. AB SHARK returns the per-arm sample size using a Welch-based two-sample formula, plus the total across arms.
What is the minimum detectable effect (MDE) in an A/B test?
The MDE is the smallest true effect your test is powered to detect with the chosen alpha and power. Smaller MDEs require quadratically larger samples. AB SHARK accepts MDE as either an absolute difference or a percentage relative to baseline.
How does stratification change required sample size?
Post-stratification reduces residual variance, so the effective sample size needed to reach the same power is smaller. The reduction depends on how much of the outcome variance the strata explain. AB SHARK supports post-stratified estimation for both binary and continuous outcomes at analysis time.
What's the difference between sample size for proportions and for means?
Proportion tests use a two-proportion z formula driven by baseline rate and lift — variance is determined by the baseline. Continuous tests use a Welch formula driven by baseline standard deviation and MDE. AB SHARK picks the right one based on the metric you choose.