What is bot traffic?

Kinds of bots

• Search engine crawlers — Googlebot, Bingbot, Baiduspider. Wanted; they decide whether you appear in search.
• AI crawlers — bots that collect training data or fetch pages to answer questions in chat assistants. See AI crawler.
• Monitoring and preview bots — uptime checks, link previews in chat apps, SEO tools.
• Scrapers — copy prices, content or contact details.
• Abusive bots — credential stuffing, fake sign-ups, click fraud, referral spam.

Some announce themselves in their user agent; many do not, and some run a real browser to look human.

How bots distort analytics

A bot that loads one page and leaves adds a visit, a visitor and a bounce. On a site with 300 human visits a day, 100 bot visits move bounce rate, average time and conversion rate noticeably. Typical signs:

• Spikes in direct traffic at odd hours, from one country or one network.
• Many visits with zero duration and identical screen sizes.
• Traffic from data centre networks rather than home or mobile connections.
• Sign-ups or form submissions with nonsense values.

How detection works

No single test catches every bot, so detection is layered. Cheap checks run first, deeper checks catch what slips through:

• User agent rules — known bot names and patterns.
• Request headers — headers real browsers always send and simple scripts forget.
• Client signals — what the page itself can observe, such as automation flags and headless-browser fingerprints.
• Network ownership — whether the visit comes from a cloud or data-centre network.
• Request rate — far more pages per minute than a person can read.

How TapCub handles bots

TapCub runs those five layers — user agent, headers, client signals, network (ASN) and rate anomalies — on every hit, and records which layers matched and how confident the result is. Detected bots are labelled with a name, an operator and a purpose (search, AI training, AI search, AI agent, monitoring and others) and kept out of human reports by default.

Bot visits are not thrown away: they have their own report and a separate stream in the realtime view, so you can see which crawlers fetch what and how often. Sites can add custom rules and a whitelist, and choose to drop bot hits entirely instead of storing them. Visits that pass the rules but look automated are flagged as suspected bots for review.

Common mistakes

• Blocking all bots. Blocking search crawlers removes you from search results.
• Trusting the user agent. Anyone can claim to be Googlebot. Verification needs more than the name.
• Filtering after the fact. Excluding bots only in some reports leaves rates inconsistent between pages.
• Ignoring sudden traffic gains. A traffic jump with no matching rise in conversions is worth checking for bots before celebrating.

Related guides: Crawl Budget, Log File Analysis for SEO and Referral Spam.

See also

• How do I import server logs?