Quick answer
Entity resolution is the process of deciding which records in one or more datasets describe the same real-world person, company, product or address, then linking or merging them. It works with exact rules, similarity scores or learned models, and is also called record linkage or deduplication.
Formula: Field weight = log2(m ÷ u) if the field agrees, log2((1 − m) ÷ (1 − u)) if not; sum the weights and compare with thresholds. Precision = correct merges ÷ all merges.
What is entity resolution?
Entity resolution is the process of deciding which records in one or more datasets describe the same real-world thing, such as the same person, company, product or address, and then linking or merging those records into one. It is needed because the same entity rarely appears the same way twice: one customer is "Ada Lovelace" in the billing system, "A. Lovelace" in the newsletter list and a bare email address in the chat log.
The same task has several names. Database researchers say record linkage, deduplication or merge-purge; Binette and Steorts open their survey by noting that integrating sources accurately is "commonly known as record linkage, de-duplication, or entity resolution". When the entity is a person, the term most marketing and analytics teams use is identity resolution. This page covers the general method; the identity page covers its application to customer profiles and cross-device tracking.
Entity resolution is not the same as entity linking, which finds mentions of known entities inside free text ("Apple" in a sentence means the company), and it is not named entity recognition, which only labels text spans as people or places. Entity resolution works on structured records and answers a yes / no / unsure question for each pair.
The four steps of an entity resolution pipeline
Most pipelines described in the literature, including the end-to-end survey by Christophides and colleagues, follow the same stages:
| Step | Question it answers | Typical technique | What goes wrong |
|---|---|---|---|
| Clean and standardize | Are values written the same way? | Lowercase emails, strip spaces and dashes from phones, expand "St." to "Street" | Skipping it makes every later step look worse than it is |
| Blocking | Which pairs are even worth comparing? | Group by a cheap key, such as the first three letters of the surname or the email domain | A key that is too strict separates true matches; too loose and nothing is saved |
| Matching | Do these two records describe one entity? | Exact rules, similarity scores, weighted probabilities or a trained model | Thresholds set by feel, with no sample to check them |
| Clustering and merge | If A matches B and B matches C, are all three one entity? | Connected components, then a rule for which value wins in the merged record | One wrong link chains two real people into one |
Blocking exists because comparing every record with every other one grows with the square of the dataset. Ten thousand records produce 10,000 × 9,999 ÷ 2 = 49,995,000 pairs. If a blocking key splits them into 500 blocks of 20 records, the work is 500 × (20 × 19 ÷ 2) = 95,000 pairs, about 0.2% of the original. The price is that two records with different keys are never compared at all, so the key must be chosen so that real duplicates share it.
Deterministic vs probabilistic matching
The matching step is where most of the argument is. There are two families, and many production systems use both.
| Deterministic (rule-based) | Probabilistic (scored) | |
|---|---|---|
| How it decides | Fixed rules: "same email" or "same email and same postal code" | Adds evidence from each field into a score, then applies thresholds |
| Output | Match or no match | A score, usually with a "review" band in between |
| Strength | Easy to explain and audit; fast; no training data needed | Handles typos, missing fields and partial evidence |
| Weakness | Misses true matches when the key differs; one shared key can merge two people | Needs an estimate of how often fields agree among matches and non-matches; harder to explain |
| Good fit | A trusted identifier exists, such as a login ID | Free-text names and addresses, mixed sources with no shared key |
The classic probabilistic model is the one published by Fellegi and Sunter in 1969, and Binette and Steorts review its modern descendants. Each field contributes a weight: log2(m ÷ u) when the field agrees and log2((1 − m) ÷ (1 − u)) when it disagrees, where m is the chance the field agrees if the records are the same entity and u is the chance it agrees by coincidence if they are different. Rare agreements, such as an identical email, are strong evidence; common agreements, such as the same first name, are weak. Learned models and embeddings replace the hand-set weights in newer systems but keep the same three outcomes: match, non-match, unsure.
Worked example: five records, two rules, one score
Sample data: a small customer list. Record A is the reference; B to E are candidates for being the same person.
| Record | Name | Phone | |
|---|---|---|---|
| A | Ada Lovelace | ada@mail.com | 555-0101 |
| B | A. Lovelace | ada@mail.com | 555-0150 |
| C | Ada Lovelace | alovelace@work.com | 555-0101 |
| D | Ada Lovell | ada.l@other.com | 555-0199 |
| E | Grace Hopper | grace@mail.com | 555-0123 |
Assume these field settings (sample values, not measured): email m = 0.90, u = 0.0001; phone m = 0.80, u = 0.001; name (after cleaning, treating "A. Lovelace" as close to "Ada Lovelace") m = 0.85, u = 0.01. The weights work out to:
| Field | Weight when it agrees | Weight when it disagrees |
|---|---|---|
| log2(0.90 ÷ 0.0001) = +13.1 | log2(0.10 ÷ 0.9999) = −3.3 | |
| Phone | log2(0.80 ÷ 0.001) = +9.6 | log2(0.20 ÷ 0.999) = −2.3 |
| Name | log2(0.85 ÷ 0.01) = +6.4 | log2(0.15 ÷ 0.99) = −2.7 |
Decision bands: score ≥ 10 is a match, score ≤ 0 is a non-match, anything between goes to a person to review.
| Pair | Phone | Name | Score | Decision | Rule "same email" says | |
|---|---|---|---|---|---|---|
| A and B | agree | disagree | agree | 13.1 − 2.3 + 6.4 = 17.2 | Match | Match |
| A and C | disagree | agree | agree | −3.3 + 9.6 + 6.4 = 12.7 | Match | No match (missed) |
| A and D | disagree | disagree | agree | −3.3 − 2.3 + 6.4 = 0.8 | Review | No match |
| A and E | disagree | disagree | disagree | −3.3 − 2.3 − 2.7 = −8.4 | Non-match | No match |
The single-key rule finds one of the two true duplicates; the scored approach finds both and sends the ambiguous Lovelace / Lovell pair to a person instead of guessing. The cost is the thresholds: moving the match line from 10 to 13 would drop A and C from automatic matches into review. Pick thresholds by hand-labelling a sample of pairs and counting wrong merges and missed merges, not by feel.
How to measure whether the resolution is good
Two numbers, computed on a hand-labelled sample of pairs, tell you most of what you need:
- Precision = correct merges ÷ all merges made. Low precision means different people were glued together, which is the more damaging error because it contaminates both profiles.
- Recall = correct merges ÷ all true duplicates. Low recall means duplicates survive, so counts such as "customers" or "users" are inflated.
For example, with 100 labelled true duplicate pairs, if a rule finds 60 of them and makes 5 wrong merges, precision = 60 ÷ 65 = 92% and recall = 60 ÷ 100 = 60%. Tightening a rule moves the two in opposite directions; the review band is what lets you accept lower automatic recall without risking precision.
Common mistakes
- Merging on a weak shared key. A family phone number or a shared office email ties different people together. Treat shared keys as evidence, not proof, unless you know they are personal.
- Transitive chains. If A matches B and B matches C, a clustering step may merge A and C even though they disagree on everything. Check cluster size; a cluster of 40 "people" is almost always an error.
- Skipping the review band. Forcing every pair into match or non-match hides the cases where a human decision would be cheapest.
- Never undoing a merge. Keep the original records and the link between them so a wrong merge can be split later, rather than overwriting sources.
- Confusing it with data matching in a warehouse. Joining two tables on an ID is not entity resolution. Resolution is needed when no reliable shared ID exists. See identity graph for a structure that stores such links and single customer view for the goal they serve.
Entity resolution in TapCub: what it does and does not do
Suppose the five records above came from a signup form, a chat transcript and a purchase log. Here is what TapCub covers.
- Deterministic binding by login ID. When your site calls
tapcub.login('u_123'), TapCub binds that login ID to the device. The first login on a device writes the user onto that session and backfills the device's earlier anonymous events, so one person on a phone and a laptop becomes one person. - Audit the merges. Open All-platform analytics › Users › Identity map. It shows people, bound devices, and a mapping log you can filter by bind, unbind, split, reject or backfill. A difference card compares visitor, device and person counts, which shows how many duplicates identity removed.
- Undo a wrong merge. Owners and admins can split a device from a person or unbind another identifier. A split reverts that session's backfill; earlier events that already carry a person ID stay as they are.
- Guard against weak keys. Values such as
guest,null,anonymous, all zeros and-1are rejected as login IDs, and a person is capped at 50 devices by default.
TapCub does not do probabilistic or fuzzy matching, and it does not clean or merge records in your CRM or billing system. Matches come only from a login ID you send. Identified mode needs Pro or above, and the console has no switch yet, so contact support to turn it on. If you need name-and-address matching across systems, do that in your own pipeline and send TapCub the resulting ID.
Sources
- Binette, O. and Steorts, R. C., (Almost) All of Entity Resolution, arXiv:2008.04443, 2020 (revised 2022).
- Christophides, V. et al., End-to-End Entity Resolution for Big Data: A Survey, arXiv:1905.06397, 2019 (revised 2020).
Was this explanation clear?
We use your answers to fix and expand this page.
Couldn’t send. Please try again in a minute.
Thanks for the feedback. We’ll use it to improve this page.You’ve already sent feedback on this page. Thanks.

Data Analyst
Explores traffic sources, conversion funnels, and retention, using clear metric definitions and concrete examples to explain what the numbers mean.