TapCub is live — analytics, insights and live chat, free on one platform
Glossary

What is bot mitigation? Signals, responses and the false-positive trade-off

Bot mitigation identifies automated traffic and limits the harm it does, without blocking real people or useful crawlers. Detection signals, response options, a numeric example of the false-positive trade-off, and what it means for your analytics.

Su He Reviewed by Ethan Brooks Also called bot management Last updated Nov 19, 2026 Glossary

Quick answer

Bot mitigation is identifying automated traffic and applying a policy to it: allow, count separately, slow down, challenge or block. The aim is to reduce harm (fraud, scraping, load, skewed numbers), not to reach zero bots, because every strict rule also stops some real people.

What is bot mitigation?

Bot mitigation is the work of recognizing automated traffic and applying a policy to it. The policy can be to allow it, count it separately, slow it down, challenge it or block it. The goal is to reduce harm, not to reach zero: scraped content, credential-stuffing attempts, fake signups, wasted server capacity and polluted analytics. Every stricter rule also stops some real people and some useful crawlers, so mitigation is a trade-off you tune, not a switch you turn on.

OWASP's Automated Threats to Web Applications project catalogs the harm in named categories such as credential stuffing, scraping, scalping and denial of inventory. Vendors often use "bot management" and "bot mitigation" interchangeably; this page uses mitigation for the act of limiting harmful automation and leaves product comparison out. For what a bot is, see bot; for how networks of compromised devices run attacks, see botnet.

Which bots are good, which are bad?

Not all automated traffic is unwanted, and the policy depends on the kind.

KindExamplesUsual policy
Good (wanted)Search engine crawlers, uptime monitors, feed readersAllow; verify they are who they say
Neutral (depends on you)SEO tool crawlers, AI crawlers and answer enginesDecide per operator; control with robots.txt
Bad (harmful)Credential stuffing, scrapers that ignore rules, fake-signup scripts, inventory hoardingLimit, challenge or block

The neutral row is where most real decisions sit. A crawler that obeys robots.txt (RFC 9309) can be steered with rules; one that ignores them needs enforcement at the server or network edge.

How are bots detected?

No single signal is reliable, so detection combines several. Reliability below is how hard the signal is for a determined bot to fake.

SignalWhat it checksReliabilityTypical failure
User-Agent stringThe name the client declaresLow: any client can claim any nameScrapers pretend to be a browser
Network addressWhether the address belongs to the operator it claimsHigh for crawlers that publish verificationResidential proxies hide scrapers
Reverse and forward DNSThe address resolves to the operator's domain and backHigh (Google documents this for Googlebot)Not available for most bots
JavaScript executionWhether the page's script runsMedium: simple bots do not run it, headless browsers doModern bots render pages
Request rate and patternRequests per second per address or sessionMediumSlow, distributed bots stay under limits
BehaviorTiming, movement, order of pagesMedium to highNeeds history; can misread assistive tools
robots.txt complianceWhether disallowed paths are fetchedOnly for crawlers that read itBad bots do not read it

Practical rule: trust a claim only when it can be verified. A request that says "Googlebot" but comes from an address that does not reverse-resolve to Google is not Googlebot, whatever the User-Agent says.

What happens after detection?

ResponseWhat it doesCost to real usersUse for
AllowLets the request throughNoneVerified good bots
Count separatelyKeeps the request but removes it from human reportsNoneAnalytics hygiene; neutral bots
Rate limitCaps requests per address or keyLow, but shared addresses can be hitScrapers, login endpoints
ChallengeAsks for proof (script check, CAPTCHA-style test)Medium: frictionSuspicious but uncertain traffic
BlockRefuses the requestHigh if wrongClear abuse
Serve decoys or slow downWastes the bot's timeLowPersistent scrapers

A sound policy escalates: count first, limit next, challenge uncertain cases, block only what is clearly abusive.

How strict should mitigation be? A worked example

Sample data: a site gets 100,000 requests a day: 88,000 from people, 3,000 from good bots and 9,000 from bad bots. Three rule sets are tried against the 9,000 bad-bot requests. A real visitor converts 3% of the time and a conversion is worth $40.

SettingBad requests stoppedBad requests leftReal visitors wrongly stoppedFalse-positive rateLost conversionsLost revenue
Lenient5,400 (60%)3,600400.05%1.2$48
Moderate7,200 (80%)1,8001800.20%5.4$216
Strict8,100 (90%)9004500.51%13.5$540

Recomputation: 450 ÷ 88,000 = 0.51%; 450 × 3% = 13.5 conversions; 13.5 × $40 = $540. Of the 8,550 requests the strict rule stopped, 8,100 were truly bad (94.7%).

The decision rule: going from moderate to strict stops 900 more bad requests (8,100 − 7,200) and wrongly stops 270 more people (450 − 180). The extra lost revenue is 270 × 3% × $40 = $324. Strict is worth it only if each bad request costs you more than $324 ÷ 900 = $0.36. For scraping that costs you almost nothing per request, moderate is the better setting; for fraud that costs $5 an attempt, strict is.

How do bots distort your analytics?

Bots do not only cost money on the server; they change your numbers. Sample data: a site records 20,000 sessions and 400 conversions, so conversion looks like 400 ÷ 20,000 = 2.00%. If 3,000 of those sessions are bots (they never convert), the human rate is 400 ÷ 17,000 = 2.35%, 17.6% higher than reported. Bots also inflate bounce rate, lower average time on page and send false traffic to the pages they scrape. Keeping bots out of the denominator is the first mitigation an analyst controls; see bot filtering.

Common mistakes

  • Aiming for zero bots: the last 10% of detection costs the most false positives.
  • Trusting the User-Agent string: it is a claim, not proof.
  • Blocking search engine crawlers by accident, then wondering why pages drop out of the index.
  • Applying one rule to every endpoint: a login form and a blog post deserve different limits.
  • Never measuring false positives: without a human-conversion check, strictness only ever goes up.
  • Treating analytics filtering as protection: removing bots from a report does not stop them using your server.

Bot mitigation and TapCub

TapCub is not a bot protection product. It does not block, challenge or rate-limit anything; it measures, so your numbers are not polluted. For the example above:

  1. Count separately by default. New projects use Detect and report separately: recognized bots appear only in Web analytics › Bots and are left out of the human overview, so the 2.35% in the example is what you see, not the 2.00%.
  2. Read who the bots are. The All bots tab lists name, operator, purpose (search, AI training, answer engines, monitoring), requests, pages visited, trust level and which detection layers matched. AI & agents shows the AI content ledger: reads per operator against visitors sent back.
  3. Understand trust. Trust is verified, suspected or unknown; unknown does not by itself mean spoofed. If a row is wrong, report a false positive.
  4. Know the limit. With only the page script, you see only bots that run JavaScript. For status codes and crawlers that do not run it, use server collection or a log import.
  5. Enforce elsewhere. Rate limits, challenges and blocks belong in your CDN, firewall or web server; use the Bots report to decide which operators to target and to check the effect afterwards.

See also: Brute Force Attack: How It Works and How to Prevent It.

See also: Credential Harvesting: What It Is, Methods and Defences.

Sources

Was this explanation clear?

We use your answers to fix and expand this page.

Su He

Data Analyst

Explores traffic sources, conversion funnels, and retention, using clear metric definitions and concrete examples to explain what the numbers mean.

FAQ

Questions people ask

Short answers to the follow-up questions that come up most.

Still have a question?

Chat with the team behind TapCub — we usually reply within a few hours on business days.

What is bot mitigation?

It is the practice of detecting automated traffic and responding with a policy such as allow, rate limit, challenge or block. The goal is to cut the damage bots cause while letting real visitors and useful crawlers such as search engines through.

What is the difference between bot mitigation and bot management?

The terms are often used interchangeably. Where they are separated, management covers the whole policy (which bots are welcome, reporting, rules), and mitigation is the act of limiting the harmful ones.

How do you detect bots?

By combining signals: whether the self-declared User-Agent checks out against the network address, whether JavaScript runs, request rate, behavior such as timing and navigation, and whether robots.txt is obeyed. No single signal is reliable alone.

Can bot mitigation block real users?

Yes. Every rule has a false-positive rate. In the sample on this page, the strictest setting stops 90% of bad-bot requests but also stops 0.51% of real visitors, so the right strictness depends on what a lost visitor and a bad request each cost you.

See this number on your own site

Add one script tag for web and apps, and the first reports fill in within minutes.

See your data clearly. Find your growth.

Every click, backed by data. Install one line of code and see your first numbers in a minute.

No credit card · Free plans for analytics and chat · Cookieless analytics