Quick answer
A confidence interval is a range of values that a sample cannot rule out for the true figure. If 120 of 4,000 visitors convert (3.0%), the 95% interval is about 2.51% to 3.58%. The 95% describes the method: repeat the study and about 95 of 100 such intervals cover the truth.
Formula: p̂ ± z × √(p̂ × (1 − p̂) ÷ n), z = 1.96 for 95% (Wald); use Wilson for small n
What a confidence interval tells you
A confidence interval (CI) turns one measured number into a range. You measured 120 conversions from 4,000 visitors, so the observed rate is 3.0%. That is the rate of the visitors you happened to get. A different 4,000 visitors would have given 2.8% or 3.2%. The interval, here 2.51% to 3.58%, is the range of true rates that this sample cannot rule out.
The "95%" describes the method, not this one interval. If you repeated the whole study many times and built an interval each time with the same recipe, about 95 of every 100 intervals would contain the true rate. Any single interval either contains it or does not, and you cannot tell which. That is why "there is a 95% chance the true rate is between 2.51% and 3.58%" is a common shorthand but not the strict meaning. For decisions, the shorthand is usually harmless, provided you remember the next point: the interval only reflects sampling noise.
Three things drive its width:
- Sample size. Width shrinks with the square root of n. Quadrupling the visitors halves the width.
- How close the rate is to 50%. Rates near 50% are noisier than rates near 0% or 100%.
- Confidence level. 99% intervals are wider than 95% intervals, because they must catch the true value more often. On the 120-of-4,000 data, the 99% Wilson interval is 2.38% to 3.78%, about 0.2 points wider than the 95% interval.
The formula and a worked conversion-rate example
For a proportion p̂ measured on n visitors, the simplest recipe (the Wald interval) is:
p̂ ± z × √(p̂ × (1 − p̂) ÷ n), with z = 1.96 for 95%
Sample data: 120 conversions out of 4,000 visitors.
- p̂ = 120 ÷ 4,000 = 0.030
- Standard error = √(0.030 × 0.970 ÷ 4,000) = √0.00000728 = 0.002697, or 0.27 percentage points
- Margin = 1.96 × 0.002697 = 0.00529, or 0.53 points
- Interval = 3.00% ± 0.53 = 2.47% to 3.53%
The Wilson interval, which the NIST/SEMATECH e-Handbook gives as the algebraic counterpart of the standard proportion test, adjusts the centre and the width. On the same data it gives 2.51% to 3.58%. For 4,000 visitors the two agree to within 0.05 points. They do not agree for small samples, as the next section shows.
The same 3.0% rate at three sample sizes (Sample data, Wilson 95%):
| Visitors | Conversions | Rate | 95% interval | Width |
|---|---|---|---|---|
| 400 | 12 | 3.0% | 1.72% to 5.17% | 3.45 pts |
| 4,000 | 120 | 3.0% | 2.51% to 3.58% | 1.07 pts |
| 40,000 | 1,200 | 3.0% | 2.84% to 3.17% | 0.33 pts |
Ten times the traffic cuts the width by a factor of 3.2, the square root of 10. To plan ahead, rearrange the Wald formula: n = z² × p × (1 − p) ÷ margin². For a rate near 3% and a margin of ±0.5 points, n = 1.96² × 0.03 × 0.97 ÷ 0.005² = 4,472 visitors. For ±0.25 points you need four times as many, about 17,886.
Why small samples need the Wilson interval
The Wald recipe assumes the sampling distribution of the rate is symmetric and bell-shaped. With few conversions it is neither. Two sample cases:
| Case | Wald 95% | Wilson 95% |
|---|---|---|
| 3 conversions out of 50 (6%) | −0.58% to 12.58% | 2.06% to 16.22% |
| 0 conversions out of 50 | 0% to 0% | 0% to 7.14% |
A negative conversion rate is impossible, and an interval of "0% to 0%" after 50 visitors claims certainty from nothing. The Wilson interval never goes below 0 and stays honest at the edges; NIST's handbook notes that its lower limit cannot be negative. Brown, Cai and DasGupta (2001) compared the standard interval with the alternatives across many sample sizes and recommended the Wilson interval for small and moderate n.
Rule of thumb: if you have fewer than about 30 conversions or fewer than 30 non-converters in the group, use Wilson (or an exact binomial interval). Above a few hundred conversions the choice barely matters.
How to read an interval when comparing two numbers
Most readers meet confidence intervals when two rates are compared: version A against version B, this month against last. Sample data: A converts 120 of 4,000 (3.0%); B converts 140 of 4,000 (3.5%).
- A's interval is 2.51% to 3.58%; B's is 2.97% to 4.12%. They overlap heavily.
- The better test is the interval of the difference. B − A = 0.50 points. Standard error of the difference = √(0.030 × 0.970 ÷ 4,000 + 0.035 × 0.965 ÷ 4,000) = 0.00396. Margin = 1.96 × 0.00396 = 0.78 points. Interval = −0.28 to +1.28 points.
- The difference interval contains 0. The data are compatible with B being worse, equal or better. The z statistic is 0.50 ÷ 0.396 = 1.26, below 1.96.
Read it with three rules:
- Interval excludes 0 (for a difference) or the target (for a single rate): the data are inconsistent with "no change" at that confidence level.
- Interval contains 0: you have not shown a difference. You also have not shown there is none; with 4,000 visitors each, differences up to about 1.3 points remain possible.
- Look at the ends, not just the middle: if even the pessimistic end is worth acting on, act; if only the optimistic end is, wait for more data.
Sample decision: B's lower end is −0.28 points, so a small loss cannot be ruled out. If losing 0.3 points would cost more than the +1.28 points at the top end is worth, collect another 4,000 visitors per version. With 8,000 each and the same rates, the margin on the difference shrinks from ±0.78 to about ±0.55 points, still not enough to separate a 0.5-point gap from zero, which tells you the real answer is a longer test or a bigger change.
Overlapping intervals do not always mean no difference, and non-overlapping ones are strong evidence; the difference interval above is the reliable check. The same logic sits behind the p-value: an interval that excludes 0 corresponds to p < 0.05 for a 95% interval.
Confidence interval compared with nearby terms
| Term | What it answers | Example from the sample |
|---|---|---|
| Confidence interval | Which true values does this sample leave plausible? | 2.51% to 3.58% |
| Margin of error | Half the interval's width | ±0.53 points |
| P-value | How surprising are these data if there were no difference? | 0.21 for B vs A (z = 1.26) |
| Statistical power | How likely is a real difference to be detected? | Low at 4,000 per arm for a 0.5-point gap |
| Prediction interval | Where will the next single observation fall? | Not the same thing; much wider |
A step-by-step method for means and proportions lives in how to calculate a confidence interval.
Choosing a method and a level before you start
Two choices are made before any data comes in, and writing them down stops the interval from being tuned after the fact. First, the method. If you expect hundreds of conversions in each group, the Wald formula is fine and easy to audit in a spreadsheet. If any group may have fewer than about 30 conversions, use Wilson from the start, because switching methods after a surprising result looks like cherry-picking. Second, the level. A 90% interval is enough to decide which of two ideas to try next when a wrong call costs a day of work. A 95% interval is the usual default for reporting. A 99% interval fits a decision that is expensive to undo, such as removing a pricing page. State the level next to every interval you publish, otherwise readers assume 95%.
A short worked rule: with 120 conversions out of 4,000, the interval half-width is 0.53 points at 95% and about 0.70 points at 99%. If the change you care about is 0.25 points, neither interval is narrow enough; you need more traffic, not a different level.
Mistakes that make intervals misleading
- Reading 95% as the probability for this one interval. It describes the procedure across repetitions.
- Treating the interval as covering every kind of error. It captures random sampling noise only. A tracking bug, bot traffic counted as visitors or a biased sample shifts the whole interval and the interval will not warn you.
- Peeking. If you recompute the interval every day and stop the first time it excludes 0, the real false-alarm rate is well above 5%. Decide the sample size before you start (the n formula above) and read the interval at that point.
- Many intervals at once. Compare 20 segments at 95% and about one will exclude the truth by chance alone.
- Using the interval on non-independent data. Counting each pageview as a trial when the same visitors return several times makes intervals too narrow. Count visitors, or sessions, consistently.
- Confusing statistical and practical significance. An interval of +0.1 to +0.3 points excludes 0 yet may be too small to matter to the business; read the size of the effect, not only its sign.
- Choosing the level after seeing the data. Decide on 90%, 95% or 99% before looking: 90% suits a quick directional read, 99% suits a decision that is expensive to reverse. State which one you used.
- Reading the interval as the range of individual outcomes. It describes where the true rate may lie, not what any one visitor will do; a 3% rate does not mean each visitor has a 3% chance in the same way.
- Rounding to a single number afterwards. Reporting "3.0%" and dropping "2.5% to 3.6%" throws away the one thing that tells the audience how much to trust it.
Reading intervals when you analyse in TapCub
Take the 3.0% versus 3.5% case. You run a sign-up change on part of your traffic and want to know whether the numbers show a real difference.
- Open All-platform analytics › Behavior › Behavior analysis and build a funnel from a landing page to your goal event. The summary gives how many entered and how many converted, so you have n and the count of conversions for the formulas above.
- Pick the period with an explicit start and end, and use the same length of period for both versions so the traffic mix is comparable. Mark the change on the chart with Configuration › Annotations.
- Compute the interval of each rate and of the difference from the entered and converted counts. TapCub gives you the counts; you do the interval arithmetic (the formulas above are short enough for a spreadsheet).
- Check the bots first. Web analytics › Bots shows crawler traffic separately, and recognised bots are kept out of visitor counts, so they are not inflating n.
- Know the boundary. TapCub does not run A/B tests or split traffic, so any version comparison needs the split to be made elsewhere. In the chat reports, Live chat › Reports › Conversion attribution shows a warning when fewer than 30 visitors chatted, which is the same small-sample concern this article covers.
Sources
- NIST/SEMATECH e-Handbook of Statistical Methods, Section 7.2.4.1 proportion defective and the Wilson interval. Wilson formula, never-negative lower limit, 4 defects in 20 units example. Checked 2026-11-21.
- Lawrence D. Brown, T. Tony Cai & Anirban DasGupta. Interval Estimation for a Binomial Proportion. Statistical Science 16(2), 2001, 101–133. Comparison of binomial interval methods.
- Edwin B. Wilson. Probable Inference, the Law of Succession, and Statistical Inference. Journal of the American Statistical Association 22(158), 1927, 209–212. Original Wilson interval.
Was this explanation clear?
We use your answers to fix and expand this page.
Couldn’t send. Please try again in a minute.
Thanks for the feedback. We’ll use it to improve this page.You’ve already sent feedback on this page. Thanks.

Data Analyst
Explores traffic sources, conversion funnels, and retention, using clear metric definitions and concrete examples to explain what the numbers mean.