Quick answer
Self-selection bias is a distortion that appears when people put themselves into a group, so the group already differed from those who did not choose it. A gap in retention or conversion then reflects who opted in as well as the effect of the thing you are studying.
What self-selection bias is, and how it differs from nearby biases
Self-selection bias appears when the people in your comparison put themselves into a group instead of being assigned to it. The ones who opted in were already different from the ones who did not, in motivation, need, skill or budget, so any gap between the groups mixes the effect of the thing you are studying with the effect of who chose it.
In product work the classic form is: "users who use feature X retain better, so X improves retention." The people who find and use X are often the ones who were going to stay anyway. The Catalogue of Bias, maintained by Oxford's Centre for Evidence-Based Medicine, describes the same pattern in medicine: in its hormone therapy example, women who chose the treatment were more health-conscious, more active and better off, which confounded the comparison.
| Self-selection bias | Selection bias (general) | Survivorship bias | |
|---|---|---|---|
| Who decides who is in the sample? | The people themselves (they opt in) | Anyone: the researcher, the process, the participants | The process of dropping out (only survivors remain) |
| Typical product example | Feature users vs non-users; beta volunteers; survey responders | Analyzing only logged-in users | Analyzing retention of users who are still active today |
| Why the gap is misleading | The groups differ before the feature exists | The sample differs from the population you care about | Those who left are missing from the data |
| Main fix | Compare people with the same starting point; randomize | Redefine the population; weight or widen the sample | Include everyone who started, not who finished |
The wider family is covered in selection bias and survivorship bias. The underlying mechanism, a third factor that drives both the choice and the outcome, is a confounding variable. This page stays on the case where the choice is the user's own.
Example 1: feature users retain better, but who chose the feature?
Sample data: 1,000 users in their first month. 300 used the reports feature at least once; 700 did not. Retention is whether a user was still active 30 days after sign-up.
| Group | Users | Retained | 30-day retention |
|---|---|---|---|
| Used the feature | 300 | 190 | 63.3% |
| Did not use it | 700 | 175 | 25.0% |
| Gap | 38.3 points |
A 38-point gap looks like proof. Now split everyone by a proxy for intent that exists before the feature: whether the user finished setup (created a project, connected a data source) on day 1.
| Day-1 setup | Group | Users | Retained | Retention |
|---|---|---|---|---|
| Finished | Used the feature | 260 | 182 | 70.0% |
| Finished | Did not use it | 140 | 91 | 65.0% |
| Not finished | Used the feature | 40 | 8 | 20.0% |
| Not finished | Did not use it | 560 | 84 | 15.0% |
Within each setup group the gap is 5 points (70 vs 65, and 20 vs 15). Weighting the two groups by their share of the 1,000 users (400 finished, 600 not) gives 0.4 × 5 + 0.6 × 5 = 5 points. So about 5 of the 38.3 points can be attributed to the feature once setup behavior is held fixed, and the rest, 33.3 points or 87%, comes from who chose it: 260 of 300 feature users had finished setup, against 140 of 700 non-users. Day-1 setup is only a proxy, so even 5 points is an upper estimate of what the feature does; other differences you cannot see remain.
Example 2: beta volunteers
Sample data: 3,000 users are invited to a beta program. 900 opt in; 2,100 do not. In the 90 days after the beta starts, 80% of volunteers are retained (720 of 900) against 50% of the others (1,050 of 2,100). That is a 30-point gap.
The same users have history. In the 90 days before the beta, 78% of the volunteers (702 of 900) and 49% of the others (1,029 of 2,100) were retained: a 29-point gap that existed before anyone saw the beta.
| Period | Volunteers | Others | Gap |
|---|---|---|---|
| 90 days before | 78% | 49% | 29 points |
| 90 days after | 80% | 50% | 30 points |
| Change in gap | +2 | +1 | +1 point |
The beta moved the gap by 1 point, not 30. Comparing a group with its own earlier behavior removes everything that was already different about the volunteers. It does not remove differences that start changing at the same moment as the beta (a volunteer who changes jobs, for example), but it is far closer than a single after-the-fact comparison.
Example 3: visitors who open live chat
Sample data: 1,800 visitors viewed the pricing page. 200 opened chat; 1,600 did not. Orders: 36 of 200 (18.0%) against 109 of 1,600 (6.8%), an 11.2-point gap. Split by intent, using whether the visitor viewed pricing two or more times:
| Intent | Opened chat | Users | Orders | Rate |
|---|---|---|---|---|
| High (2+ pricing views) | Yes | 150 | 30 | 20.0% |
| High | No | 450 | 63 | 14.0% |
| Low (1 view) | Yes | 50 | 6 | 12.0% |
| Low | No | 1,150 | 46 | 4.0% |
Inside each intent group the gap is 6 and 8 points instead of 11.2. Three quarters of the people who opened chat (150 of 200) were high-intent, against 450 of 1,600 non-chatters (28%). Part of the gap is intent; some may be real. A measured gap after splitting is an estimate to test, not a result to report.
How to reduce self-selection bias
| Method | What you do | What it fixes | What it does not fix |
|---|---|---|---|
| Compare like with like | Split users by a trait fixed before the choice (source, plan, day-1 behavior, signup week), compare inside each split, combine the results | Differences you measured | Differences you did not measure |
| Compare before and after | Look at the same people's behavior before and after, and compare the change between groups | Stable differences between groups | Differences that change at the same time as the exposure |
| Delay the comparison | Start measuring from a fixed point (first week) rather than from "first use" | Early dropouts looking like non-users | Motivation |
| Randomize | Assign the feature at random and compare | Everything, if the groups are large enough | Needs an experiment tool |
Two habits help regardless of method. Write down the choice that put people in each group before looking at outcomes, and ask what else would drive that choice. And keep each comparison small enough to read: after three or four splits the groups are too small to trust.
Common mistakes
- Reading a raw gap as a feature effect. The 38.3-point gap in Example 1 is a description of two groups, not of the feature.
- Adjusting for something that happens after the feature. Splitting by "completed three sessions" when the feature itself causes sessions removes part of the real effect. Use traits fixed before exposure.
- Counting the volunteers as the population. A survey, a beta or a feedback widget tells you about people who answered, not about everyone who used the product.
- Treating one adjustment as proof. Matching on setup behavior (Example 1) still leaves unobserved differences; say so in the report.
- Splitting until the groups are tiny. With 40 users in one cell (Example 1, not finished), one extra retained user moves the rate 2.5 points.
Self-selection bias in TapCub
Take Example 1 into TapCub. The goal is to find out whether people who use a feature stay longer than comparable people who do not, and how much of the gap survives when you compare like with like.
- Record the choice as an event. Send an event when a user first uses the feature (for instance
report_viewed) so you can build a rule segment for people who did it and people who did not. - Build the groups as segments. In All-platform analytics › Users › Segments & tags, create rule segments such as "first seen in the last 60 days and did
report_viewed" and "first seen in the last 60 days and did not doreport_viewed". - Read retention per group. In All-platform analytics › Behavior › Behavior analysis, open Retention with the sign-up as the start and apply one segment at a time. Retention has no breakdown, so you run it once per segment and compare the rates.
- Split by something fixed beforehand. Add a second condition to each segment, such as acquisition channel (from Web analytics › Sources), new vs returning, or whether the user did a setup event on day 1, and repeat. If the gap shrinks from 38 points to 5, the first number was mostly who chose the feature.
- Know the limit. TapCub does not run A/B experiments or assign users to groups, so it cannot randomize. It shows the gap and how much of it survives a comparison; to settle causation you would run the experiment in your own tool and measure the result here.
Sources
- Heckman, J. J. (1979). Sample Selection Bias as a Specification Error. Econometrica, 47(1), 153–161.
- Catalogue of Bias. Selection bias (hormone therapy example of self-selection and confounding).
Was this explanation clear?
We use your answers to fix and expand this page.
Couldn’t send. Please try again in a minute.
Thanks for the feedback. We’ll use it to improve this page.You’ve already sent feedback on this page. Thanks.

Technical Engineer
Focuses on tracking code, SDKs, and API integrations, with practical guidance on implementation checks and troubleshooting.