Quick answer
The minimum detectable effect (MDE) is the smallest true difference between variants that an experiment is designed to detect at a chosen significance level and power. A smaller MDE needs a larger sample; with the same traffic, the test has to run longer.
Formula: n per group ≈ (z₁₋α/₂ + z₁₋β)² × [p₁(1−p₁) + p₂(1−p₂)] ÷ (p₂ − p₁)²
What is minimum detectable effect?
The minimum detectable effect (MDE) is the smallest real difference between two variants that your experiment is set up to catch. You choose it before the test, together with the significance level (often 5%) and power (often 80%). Those three numbers decide how many visitors you need.
If the true effect is at least the MDE, the test will usually detect it (80% of the time at 80% power). If the true effect is smaller, the test may miss it.
Relative vs absolute MDE
| Baseline | Relative MDE | Absolute change | Target rate |
|---|---|---|---|
| 5.0% | 10% | 0.5 percentage points | 5.5% |
| 5.0% | 20% | 1.0 percentage point | 6.0% |
| 40% | 10% | 4 percentage points | 44% |
Testing tools differ in which one they ask for. Mixing them up changes the sample size by a large factor, so always say which you mean.
Formula
For conversion rates, the sample size per group is approximately:
n ≈ (z₁₋α/₂ + z₁₋β)² × [p₁(1−p₁) + p₂(1−p₂)] ÷ (p₂ − p₁)²- p₁ is the baseline rate, p₂ the rate you want to be able to detect.
- With α = 0.05 (two-sided) and 80% power, (1.96 + 0.84)² ≈ 7.85.
The key point: n grows with 1 ÷ (p₂ − p₁)². Halving the MDE needs about four times the sample.
Worked example
Baseline conversion 5%, α = 0.05 two-sided, 80% power, and 4,000 visitors per group per day (sample traffic):
| Relative MDE | Target rate | Visitors per group | Days needed |
|---|---|---|---|
| 5% | 5.25% | 122,121 | 30.5 |
| 10% | 5.50% | 31,231 | 7.8 |
| 20% | 6.00% | 8,155 | 2.0 |
A team that only cares about lifts of 10% or more can finish in about a week. Detecting a 5% lift takes a month. Even when the 20% test could finish in two days, run at least one full week so weekday and weekend behaviour are both included.
How to choose an MDE
- Start from the decision. What is the smallest lift that would justify shipping and maintaining the change? That is your MDE.
- Check traffic. Use the formula or a calculator. If the test would take months, test a bolder change, pick a metric closer to the change, or accept a larger MDE.
- Fix it in advance. Write down MDE, significance level, power and test length before launch, along with any guardrail metrics.
Common mistakes
- Choosing the MDE after seeing data. It turns planning into rationalising.
- Stopping early when the p-value dips below 0.05. Peeking inflates false positives; run to the planned sample.
- Reading a non-significant result as "no effect". It only means no effect at least as large as the MDE was found with the planned confidence.
- Ignoring imbalance between groups. A confounding variable or a broken split can create or hide differences.
Getting the baseline in TapCub
TapCub does not run A/B tests, but it gives you the inputs for planning one. Your baseline conversion rate and daily traffic come from Web analytics › Goals or a funnel in All-platform analytics. Use the last four full weeks to smooth out weekday effects, then plug the rate and visitors per day into the formula above.
Was this explanation clear?
We use your answers to fix and expand this page.
Couldn’t send. Please try again in a minute.
Thanks for the feedback. We’ll use it to improve this page.You’ve already sent feedback on this page. Thanks.
Product Analyst
Focuses on event analysis, conversion funnels, and retention, using clear metric definitions and structured analysis to explain product performance.