"How long should I run my A/B test?" It's the most common question in experimentation. And most people get it wrong — either stopping too early or running way too long.
Duration Calculator
The smallest improvement you want to detect (e.g., 20% = detecting a change from 3% to 3.6%)
Sample Size Needed
25,350
Estimated Duration
26 days
Approximately
4 weeks
Why Most People Get It Wrong
Mistake #1: Stopping when you see a winner
Early results are unreliable. A test showing 95% confidence on day 2 might flip completely by day 7. Always run for the full calculated duration.
Mistake #2: Ignoring day-of-week effects
Behavior differs by day. Always run for at least 1 full week (ideally 2) to capture these variations.
Mistake #3: Expecting to detect tiny changes
Detecting a 5% improvement requires 10x more sample size than detecting a 50% improvement. Be realistic about what you can detect.
The Math (Simply Explained)
Test duration depends on four factors:
Traffic
More visitors = faster tests. Double your traffic, halve your duration.
Baseline Conversion Rate
Higher conversion rates need less sample size. A 10% rate tests faster than a 1% rate.
Minimum Detectable Effect
Smaller effects need bigger samples. Detecting 5% lift is much harder than 50%.
Number of Variants
More variants = more sample needed. Each variant splits your traffic further.
Rules of Thumb
Minimum: 1 week
Never run a test for less than 7 days, regardless of sample size.
Sweet spot: 2-4 weeks
Most tests should run 2-4 weeks for reliable results.
Maximum: 6-8 weeks
If a test needs more than 8 weeks, consider testing a bigger change.
What If You Don't Have Enough Traffic?
The Bottom Line
Calculate your required duration before starting. Commit to that duration. Don't peek and stop early. The discipline to run proper tests is what separates teams that learn from teams that just guess.
Use the calculator above to plan your tests, and remember: patience is a competitive advantage.