Quick answer
Causation means that changing A makes B change, and B would have been different had A not changed. It needs more than two metrics moving together: A must come first, other explanations must be ruled out, and the strongest proof is a randomised experiment.
Formula: Causal effect = outcome with A − outcome without A, for the same users at the same time
What does causation mean?
Causation means that changing one thing makes another thing change, and that the change would not have happened otherwise. In the usual statistical wording: A causes B if, for the same people at the same time, B would have been different had A been different. That "had been different" clause is the counterfactual idea that philosophers use to define causes (see the Stanford Encyclopedia entry in Sources), and it is why causation is hard to see in data. For any one user you only ever observe one world: the one where they used the feature, or the one where they did not.
Two practical consequences follow:
- Most causes are probabilistic. A cause raises the chance of an effect rather than guaranteeing it. Suppose a new onboarding checklist raises week-1 retention from 40% to 44% (hypothetical numbers): it is a cause even though most users who see it would have stayed, or left, anyway.
- Data alone cannot tell you the direction. Two metrics that move together are a correlation. Whether A drives B, B drives A, or something else drives both is a separate question that needs a design, not more rows. The side-by-side comparison of the two ideas is in correlation vs causation; this page stays with causation itself.
What do you need before calling something causal?
Three conditions are the minimum, and they are cumulative:
- Association. A and B move together. Without this there is nothing to explain, but it is the cheapest condition and the one most often mistaken for the whole case.
- Time order. A happens before B. If retention in month 2 is the outcome, the feature must have been used before month 2 ended. Habits and "became a power user" often blur this: users who were already staying can use more features because they stayed.
- No better explanation. Every other plausible reason for the association has been ruled out, measured and held fixed, or made irrelevant by randomising who gets A.
The strongest way to satisfy the third condition is an experiment in which a coin flip decides who gets A. Short of that, the evidence is graded:
| Evidence | What it shows | Weakness |
|---|---|---|
| Raw comparison of A-users and non-A-users | Association only | Anyone can pick A for reasons that also drive B |
| Comparison inside strata (channel, device, plan) | Association survives the strata you measured | Unmeasured differences remain |
| Before and after, with a comparison group | Change over time, net of a shared trend | Needs a group that would have moved the same way |
| Natural experiment (an arbitrary rollout rule) | Near-random assignment you did not control | Rare, and the rule must really be arbitrary |
| Randomised experiment | Assignment is independent of everything else | Costs traffic and time; sometimes impossible |
In 1965 the epidemiologist Austin Bradford Hill listed nine considerations for judging whether an association is causal: strength, consistency, specificity, temporality, a dose-response gradient, plausibility, coherence, experimental evidence and analogy. He stressed that none of them is a test that settles the question on its own. They are things to look for, which is also the right way to treat the checklist in this article.
What kinds of causal relationship can show up in product data?
When two metrics move together, four stories can explain it. Only the first is causation from A to B in the sense you probably want.
| Pattern | Story | Product example |
|---|---|---|
| Direct cause | A changes B | Faster page load leads to more completed checkouts |
| Mediated cause | A changes M, and M changes B | A discount email raises visits, and visits raise orders |
| Common cause | C changes both A and B, A and B do not affect each other | Organic visitors both use search more and buy more |
| Reverse cause | B changes A | People who are about to churn stop opening the app, so "inactive" predicts churn without causing it |
Mediation still counts as causation, but it changes what you can promise: if the discount email stops raising visits, the orders disappear too. The common-cause and reverse patterns are the usual reasons that a number looks like an effect and is not. Separating them is mostly a matter of listing candidate common causes (traffic source, device, country, plan, how long the user has been around, purchase intent) and then checking the association inside each.
Worked example: does using a feature cause retention?
Sample data: 2,000 new users in one month. 800 used a new reports feature in their first week; 1,200 did not. Retention means active again in week 4.
| Group | Users | Retained | Retention |
|---|---|---|---|
| Used the feature | 800 | 414 | 51.75% |
| Did not use it | 1,200 | 400 | 33.33% |
| Difference | +18.4 points |
Taken at face value, the feature "adds 18 points" of retention. Now split by acquisition channel, which is a plausible common cause: organic users arrive with intent and stay more, and they are also more likely to explore features.
| Channel | Group | Users | Retained | Retention |
|---|---|---|---|---|
| Organic (1,000 users) | Used the feature | 600 | 372 | 62.0% |
| Organic | Did not use it | 400 | 240 | 60.0% |
| Paid (1,000 users) | Used the feature | 200 | 42 | 21.0% |
| Paid | Did not use it | 800 | 160 | 20.0% |
Inside each channel the gap is +2.0 points (organic) and +1.0 point (paid). Weighting each channel by its share of the 2,000 users (50% each) gives an adjusted gap of 0.5 × 2.0 + 0.5 × 1.0 = +1.5 points. About 92% of the original 18.4 points came from who used the feature, not from the feature: 600 of the 800 feature users were organic, against 400 of the 1,200 non-users.
Two things are worth taking from it. First, the adjusted +1.5 points is still not proof of causation: it controls for one variable, and users who explore reports may differ in ways channel does not capture. Second, a gap of 1.5 points on groups this size needs its own check for noise; the arithmetic for that is in statistical significance.
How do you check a causal claim in practice?
Run the claim through five questions, in this order, and stop where it fails:
- Which came first? Define the outcome window so it starts after the cause. If "used the feature" is measured over the same weeks as "retained", fix the order: feature use in days 0–7, retention in days 22–28.
- Who chose A? If users chose it themselves, assume they differ. If a rule assigned it (a staged rollout by account ID, a region launch), ask whether the rule was arbitrary.
- What else differs between the groups? List at least five candidates and compare the groups on each. If channel mix, device mix or plan mix differs strongly, the raw comparison is unsafe. See confounding variable and Simpson's paradox for how a split can shrink or even flip a gap.
- Does the gap survive splitting? If the gap holds in every stratum with the same sign and a similar size, the claim is stronger. If it vanishes in some strata, that is where to look.
- Can you randomise, or hold out? If the decision is large enough to be costly if wrong, run an experiment, or keep a random holdout group that never receives the change. The related idea of measuring what would have happened anyway is incrementality.
What are the common mistakes?
- Reading a correlation coefficient as an effect size. A correlation of 0.6 says the two move together tightly; it says nothing about how much B changes if you change A.
- Comparing opt-in groups. Users who turn on a feature, open emails or join a loyalty programme chose to. That choice is the selection bias behind most "feature X lifts retention" charts.
- Controlling for the wrong variable. Splitting by something that sits between A and B (for example, "visits" when the discount works through visits) removes part of the real effect.
- Treating attribution as proof. A report that credits a channel with a share of conversions allocates credit by rule; it does not show that those conversions would have been lost without the channel.
- Stopping at "no correlation". A real cause can be hidden by a common cause pushing the other way, or by noise in a small sample.
- Declaring causation from one split. One stratified table narrows the explanations; it does not remove the unmeasured ones.
How do you use TapCub to check a claim like this?
Take the reports-feature example above. TapCub can give you the stratified view quickly; it does not run the experiment that would settle it.
- Define "used the feature" once. In All-platform analytics › Users › Segments & tags, create a rule segment from the feature's event with a time window (did the event in the last 30 days, for example). Segments are available as a group in other analyses, so the same definition is used everywhere. Anonymous visitors have no profile and never join a segment, so identify logged-in users first.
- Compare outcomes by channel. In All-platform analytics › Behavior › Behavior analysis, build a funnel from the signup event to a week-4 return event and break it down by channel. Breakdown keeps the 10 groups with the most entries. Retention analysis has no breakdown, so to see retention by channel, filter it to one channel at a time or use a segment.
- Ask which dimensions moved the metric. In All-platform analytics › Operations › AI Assistant, the Why did it change tab splits a change in a metric across channel, referrer domain, country, device, entry page and similar dimensions and shows each value's contribution to the change. That is a decomposition of what moved, not a test of why; it points you at candidate common causes to check.
- Mark the launch. In Configuration › Annotations, add the release date so before-and-after comparisons are not run blind.
- Run the experiment elsewhere. TapCub does not run A/B experiments or feature flags and does not do the random assignment; it only measures. If you run an experiment with your own tool, send the variant name as an event property and compare the groups here.
See also: What is correlation? The correlation coefficient explained.
Sources
- Stanford Encyclopedia of Philosophy, Counterfactual Theories of Causation.
- Stanford Encyclopedia of Philosophy, Probabilistic Causation.
- Hill, A. B. (1965). The Environment and Disease: Association or Causation? Proceedings of the Royal Society of Medicine, 58(5), 295–300.
- Pearl, J. (2009). Causal inference in statistics: An overview. Statistics Surveys, 3, 96–146.
Was this explanation clear?
We use your answers to fix and expand this page.
Couldn’t send. Please try again in a minute.
Thanks for the feedback. We’ll use it to improve this page.You’ve already sent feedback on this page. Thanks.

Technical Engineer
Focuses on tracking code, SDKs, and API integrations, with practical guidance on implementation checks and troubleshooting.