Free tool Traffic change significance: real drop or random noise?
Enter visits (and optionally conversions) for two periods. The calculator tests whether the change is larger than chance alone would produce, shows the range you would expect if nothing had changed, and with daily data accounts for weekday patterns and the day-to-day swings real sites have.
- Exact test, checked against SciPy
- Weekday-adjusted with daily data
- Runs locally
Compare two periods
The sample is a store whose visits fell 9.6% week over week. With totals alone the drop looks highly significant; switch to the daily-data example to see what day-to-day variation does to that conclusion.
A change counts as significant when p is below 1 − this level.
Sessions or visits, a whole number.
Periods of different length are compared per day.
Optional. Orders, sign-ups or any goal.
Optional.
One day per line: date, visits[, conversions], or just the visits. Oldest first; an exported report with a header row works.
The N days before it are the previous period; all earlier days are used as the baseline for weekday patterns and variation.
Change in visits
−9.6%
Per day when the periods differ in length.
p-value (Poisson)
< 0.001
Assumes every visit is independent.
p-value (adjusted)
—
Allows for the day-to-day variation in your data.
Dispersion φ
—
Day-to-day variance ÷ what pure chance gives. 1 = Poisson.
Expected range with no change
12,562 – 13,192
Where the current period would land in most weeks if the true rate had not moved.
Confidence interval of the change
−11.8% … −7.3%
All tests
| Metric | Previous | Current | Change | Test | p | Result |
|---|---|---|---|---|---|---|
| Visits | 12,877 | 11,643 | −9.6% | Exact Poisson | 3.39 × 10⁻¹⁵ | Significant |
| Conversions | 311 | 288 | −7.4% | Exact Poisson | 0.3687 | Within noise |
| Conversion rate | 2.42% | 2.47% | +2.4% | Two-proportion z | 0.7673 | Within noise |
Weekday pattern in the baseline
Average visits per weekday and their ratio to the overall average.
Assumptions and warnings
- NoteWith thousands of visits the Poisson test flags even small changes. Real traffic varies more from day to day than the test assumes; paste daily data to check.
Share these inputs: the values go after the # in the link, so they never reach a server.
https://tapcub.com/tools/traffic-change-significance
Version v1.0Updated Update history
What "significant" means for traffic
A significance test asks one question: if the underlying rate had not changed, how surprising would this difference be? It does not tell you why traffic moved.
p-value
The probability of a change at least this large if the true rate stayed the same. Below 5% (at 95% confidence) the change counts as significant. It is not the probability that the change is real.
Expected range
Where the current period would land in most weeks with no real change. A result inside it is ordinary; outside it is worth a look. With daily data the range widens to match your site's actual variation.
Dispersion φ
How much more daily visits swing than coin-flip randomness. Real sites usually sit well above 1 because of campaigns, news and weekdays; the higher φ, the less a single week means.
Conversion rate vs. conversions
Fewer conversions with fewer visits is expected. The conversion-rate row asks whether visitors converted less often, which is usually the more useful signal.
Independence is the key assumption: the Poisson test treats every visit as a separate random event. Returning visitors, bots, campaigns and outages break it, which is why the adjusted result matters more than the plain one.
Check a change before you react to it
Use it when a dashboard shows a drop and someone asks what happened.
-
1
Pick comparable periods
Whole weeks against whole weeks, or the same weekdays. Avoid holidays, launches and campaigns in one period but not the other.
-
2
Enter totals, or paste daily data
Totals give a quick answer. Daily data from your analytics export (five weeks or more) let the tool measure your site's normal variation and weekday pattern.
-
3
Read the adjusted row first
If the change survives the adjustment, look for a cause: channels, landing pages, devices. If it does not, wait for another week of data before acting.
The tests behind the numbers
Visits are counts, so the main test treats them as Poisson. Daily data add an estimate of how much your traffic varies beyond that.
- The exact test conditions on the total n = x₁ + x₀: if nothing changed, x₁ ~ Binomial(n, t₁ ⁄ (t₁ + t₀)). The two-sided p-value adds up all outcomes no more likely than the one observed, as R's
poisson.testand SciPy'sbinomtestdo; results match SciPy 1.13 to about ten significant digits - φ = Σ (x − e)² ⁄ e ⁄ (days − groups), where e is the baseline average for that weekday (quasi-Poisson; McCullagh & Nelder, Generalized Linear Models, 2nd ed., 1989); φ below 1 is treated as 1
- Weekday groups come from the dates; without dates, days are grouped by their distance from the last day modulo 7
Change
(x₁ ⁄ t₁) ⁄ (x₀ ⁄ t₀) − 1
x = visits, t = days; 1 = current, 0 = previous
Exact p-value
Σ P(X = k) over k with P(X = k) ≤ P(X = x₁)
X ~ Binomial(x₀ + x₁, t₁ ⁄ (t₀ + t₁))
Expected range
E ± z·√(φ·(E + r²·x₀)), E = r·x₀, r = t₁ ⁄ t₀
z = 1.96 at 95%; φ = 1 without daily data
Adjusted z
ln(rate ratio) ⁄ √(φ·(1 ⁄ x₀ + 1 ⁄ x₁))
two-sided p from the normal distribution
Weekday-adjusted z
(x₁ − E_w) ⁄ √(φ·(E_w + E_w ⁄ w))
E_w = sum of baseline weekday averages for the current days; w = baseline days per weekday
Conversion rate z
(p₁ − p₀) ⁄ √(p̄(1 − p̄)(1 ⁄ n₀ + 1 ⁄ n₁))
p̄ = pooled rate
A 9.6% drop, three ways
The store in the sample had 12,877 visits last week and 11,643 this week. The same five weeks of daily data are in the second example.
| Method | Change | p | Verdict |
|---|---|---|---|
| Exact Poisson on the totals | −9.6% | 3.4 × 10⁻¹⁵ | Significant |
| Quasi-Poisson, φ = 17.1 from 28 baseline days | −9.6% | 0.057 | Within noise at 95% |
| Against the weekday baseline of the previous four weeks | −4.0% | 0.35 | Within noise |
| Conversion rate 2.42% → 2.47% | +2.4% | 0.77 | Within noise |
Last week happened to be the strongest of the five, so part of the "drop" is the previous week being unusually high. Visitors converted at the same rate. The right reaction is to keep watching, not to rewrite the landing pages.
How week-over-week comparisons mislead
Most false alarms come from the comparison, not from the traffic.
Comparing unequal weekdays
A Monday–Sunday week against a period with an extra weekend day, or a month with five Saturdays against one with four. Compare whole weeks or the same weekdays.
Trusting the plain Poisson p-value
With thousands of visits it calls almost every change significant. Real traffic varies far more than the Poisson model assumes; use daily data to measure by how much.
Testing every day
Checking significance daily and reacting to the first "significant" result finds false alarms. Decide what you will compare before you look, or rely on alerts with a fixed rule.
Ignoring what is in the count
A spike from a bot or referrer spam is significant and meaningless. Check the source of a change before reading anything into it.
Get told when a change is unusual
Instead of checking dashboards and running tests, TapCub watches the metrics and notifies you.
Drop, spike and anomaly alerts
Rules on any metric against the previous period, a threshold, or an anomaly against the seasonal baseline, sent to email, WeCom, DingTalk, Feishu or a webhook.
Learn moreHumans and bots apart
Bots are counted separately, so a crawler burst does not show up as a traffic spike in human reports.
Learn moreChanges broken down by channel
Compare periods by source, channel and landing page to see where a change came from.
Learn moreTraffic change questions
Still have a question?
Chat with the team behind TapCub — we usually reply within a few hours on business days.
My traffic dropped 10% this week. Should I worry?
Enter both weeks here. With totals only, a 10% drop on a few thousand visits usually looks significant, because the test assumes each visit is independent. Paste five weeks of daily data and the tool measures your normal variation; on many sites a 10% week-to-week move turns out to be ordinary.
How is statistical significance calculated for traffic?
Visit counts are treated as Poisson counts. Given the total of both periods, the current period's share follows a binomial distribution if nothing changed, and the p-value is the probability of a share at least as extreme as the one observed. This is the standard exact test for comparing two Poisson rates.
What is a normal week-to-week traffic fluctuation?
It depends on your site, which is why the tool does not give a fixed number. The dispersion φ from your daily data is the answer for your site: φ = 1 means only random noise, and real sites are usually well above that because of weekdays, campaigns and news.
Why does a small drop count as significant on a big site?
Random noise in a Poisson count grows with the square root of the count, so with 100,000 visits a 2% change is far outside pure chance. Real traffic has more variation than that, which is what the adjusted result accounts for.
Should I compare with last week or last year?
Last week reacts fastest but carries weekday and campaign noise; the same week last year handles seasonality but your site has changed since. Use whole weeks, and for seasonal businesses look at both.
Why is conversion rate tested separately?
Fewer visits normally means fewer conversions. The conversion-rate test asks whether the people who came converted less often, which separates a traffic problem from a site or offer problem.
What does this test not account for?
Holidays, seasonality beyond the weekday pattern, trends, campaigns, tracking changes and returning visitors making several visits. The weekday adjustment needs at least two weeks of baseline, and it cannot tell you why traffic changed.
Is my data uploaded?
No. The calculation runs in your browser. The share link carries the two totals only; pasted daily data never leave the page.
Update history
New features, fixes and data refreshes for this tool, newest first.
-
· v1.0
LaunchedFirst release.
Missing something or found an error? Tell us — requests that come up often go into the next update. All tool updates
Let alerts do the checking
TapCub counts humans and bots separately and alerts you when a metric moves outside its usual range, so you are not testing every dip by hand.