TapCub is live — analytics, insights and live chat, free on one platform

Free tool Crawler IP range checker: which log addresses are real crawlers?

Search engines and AI companies publish the IP ranges their crawlers use. Paste a list of addresses — or raw lines from an access log — and see which ones sit in an official range, whose it is, and which matching prefix it falls in. 23 lists from 11 operators, 5,966 prefixes, fetched 2026-11-17. Nothing leaves your browser.

  • 23 official lists
  • Up to 5,000 addresses
  • Runs locally, nothing sent

Check addresses against official crawler ranges

One address per line, or paste log lines: the first IP address on each line is used. Duplicates are counted once.

Check

IPv4, IPv6, [2001:db8::1]:443, CSV rows and Nginx / Apache log lines all work. Lines starting with # are skipped.

Addresses

9

Unique, after removing duplicates

On an official list

7

Not on any list

2

Unreadable lines

0

Summary

  • OK7 of 9 addresses are on an official crawler list.
  • NoteNot on any list: 1. That does not make them fake: most are people, and some operators (Baidu, Yandex) publish no list. Check claimed crawlers one by one with the fake bot checker.
  • WarningPrivate or reserved addresses: 1. Real crawlers never come from these; your logs probably record a proxy instead of the client.

By crawler list

One address can be on more than one list (OpenAI uses some ranges for two crawlers).

OperatorCrawlersTypeAddresses
GoogleGooglebot, Google-InspectionTool, GoogleOther, Storebot-GoogleSearch engine2
MicrosoftBingbotSearch engine1
AppleApplebotSearch engine1
OpenAIGPTBotAI crawler1
AnthropicClaudeBot, Claude-User, Claude-SearchBotAI crawler1
AhrefsAhrefsBot, AhrefsSiteAuditSEO tool1

Addresses

AddressResultOperator · crawlerOfficial prefixLine
66.249.66.1Official crawlerGoogle · Googlebot, Google-InspectionTool, GoogleOther, Storebot-Google66.249.66.0/271
157.55.39.1Official crawlerMicrosoft · Bingbot157.55.39.0/242
20.171.207.5Official crawlerOpenAI · GPTBot20.171.207.0/243
216.73.216.12Official crawlerAnthropic · ClaudeBot, Claude-User, Claude-SearchBot216.73.216.0/224
17.241.208.161Official crawlerApple · Applebot17.241.208.160/275
2001:4860:4801:10::1Official crawlerGoogle · Googlebot, Google-InspectionTool, GoogleOther, Storebot-Google2001:4860:4801:10::/646
54.36.148.1Official crawlerAhrefs · AhrefsBot, AhrefsSiteAudit54.36.148.0/237
203.0.113.77Documentation——8
2a03:2880:f800::1Not listed——9
CSV
ip,result,operator,crawler,prefix,source_file
66.249.66.1,official,Google,"Googlebot, Google-InspectionTool, GoogleOther, Storebot-Google",66.249.66.0/27,https://developers.google.com/static/crawling/ipranges/common-crawlers.json
157.55.39.1,official,Microsoft,Bingbot,157.55.39.0/24,https://www.bing.com/toolbox/bingbot.json
20.171.207.5,official,OpenAI,GPTBot,20.171.207.0/24,https://openai.com/gptbot.json
216.73.216.12,official,Anthropic,"ClaudeBot, Claude-User, Claude-SearchBot",216.73.216.0/22,https://claude.com/crawling/bots.json
17.241.208.161,official,Apple,Applebot,17.241.208.160/27,https://search.developer.apple.com/applebot.json
2001:4860:4801:10::1,official,Google,"Googlebot, Google-InspectionTool, GoogleOther, Storebot-Google",2001:4860:4801:10::/64,https://developers.google.com/static/crawling/ipranges/common-crawlers.json
54.36.148.1,official,Ahrefs,"AhrefsBot, AhrefsSiteAudit",54.36.148.0/23,https://api.ahrefs.com/v3/public/crawler-ip-ranges
203.0.113.77,doc,,,,
2a03:2880:f800::1,not_listed,,,,

One row per address with the matching list’s file URL, for your firewall or log tooling.

Runs entirely in your browser. The official lists are a static file on tapcub.com; the addresses you paste are never uploaded or stored.

Version v1.0Updated Update history

Read the result

What a match does and does not mean

An official list proves identity in one direction only.

On a list: it is that crawler

The operator says requests from this range are theirs. Combined with the matching user agent, this is as strong as the DNS check. Allow it if you want to be crawled.

Not on a list: unknown, not fake

Most addresses in a log are people, so “not listed” is normal. It only means fake when the request also claims to be one of the listed crawlers.

Several lists at once

Some ranges serve more than one crawler: OpenAI shares ranges between GPTBot and OAI-SearchBot, and DuckDuckGo publishes one list for both of its bots. All matches are shown.

Private or reserved

An address like 10.0.0.5 or 192.168.1.20 means your server sees a proxy, not the client. Fix the logging (X-Forwarded-For, CF-Connecting-IP) before checking bots.

The lists change every few weeks. This page uses the copy fetched on 2026-11-17; download the operator’s file for the current one.

How to use it

Three steps

No sign-up. Nothing is uploaded.

  1. 1

    Paste addresses or log lines

    Up to 5,000 unique addresses. Raw access log lines work: the first IP on each line is taken.

  2. 2

    Read the summary

    How many are official crawlers, from which operators, and which lines could not be read.

  3. 3

    Filter and export

    Show only matches or only unlisted addresses, then copy or download the CSV.

How it works

Official lists, matched locally

Every operator below publishes a JSON file of CIDR prefixes. TapCub downloads them with a script and ships them as a static file; your browser does the matching.

  • Each line is scanned for the first IPv4 or IPv6 address; brackets, ports and zone IDs are removed and IPv4-mapped IPv6 is treated as IPv4
  • Each address is tested against every prefix: it matches when its first n bits equal the prefix’s (RFC 4632). Lookups use a sorted table, so 5,000 addresses take a few milliseconds
  • When one list has several prefixes containing the address, the most specific one is shown
  • Private, loopback, documentation and other special-purpose addresses (RFC 6890) are flagged separately
  • The CSV lists the source file of each match so you can audit it
Official lists in this tool (fetched 2026-11-17)
OperatorCrawlersSource file
GoogleGooglebot and other common crawlers, special crawlers, user-triggered fetchers and agentsdevelopers.google.com/static/crawling/ipranges/*.json (5 files)
MicrosoftBingbotbing.com/toolbox/bingbot.json
OpenAIGPTBot, OAI-SearchBot, ChatGPT-User, OAI-AdsBotopenai.com/gptbot.json and 3 more
AnthropicClaudeBot, Claude-User, Claude-SearchBotclaude.com/crawling/bots.json
AppleApplebotsearch.developer.apple.com/applebot.json
PerplexityPerplexityBot, Perplexity-Userperplexity.ai/perplexitybot.json, perplexity-user.json
DuckDuckGoDuckDuckBot, DuckAssistBotduckduckgo.com/duckduckbot.json, duckassistbot.json
AmazonAmazonbot, Amzn-SearchBot, Amzn-Userdeveloper.amazon.com/amazonbot/… (3 pages)
Mistral AIMistralAI-User, MistralAI-Indexmistral.ai/mistralai-*-ips.json
Common CrawlCCBotindex.commoncrawl.org/ccbot.json
AhrefsAhrefsBot, AhrefsSiteAuditapi.ahrefs.com/v3/public/crawler-ip-ranges
Example

Six lines from an access log

Click “Try: access log lines” to load them. All six claim to be crawlers in their user agents.

Log lines matched against the lists fetched 2026-11-17
AddressUser agent saysResult
66.249.66.1GooglebotGoogle common crawlers, 66.249.66.0/27
157.55.39.1bingbotBingbot, 157.55.39.0/24
20.171.207.5GPTBotGPTBot, 20.171.207.0/24
203.0.113.77GooglebotDocumentation address: cannot be a real crawler. In a real log, an address that is not on Google’s list while claiming Googlebot is fake
216.73.216.12ClaudeBotAnthropic, 216.73.216.0/22
2001:4860:4801:10::1GooglebotGoogle common crawlers, 2001:4860:4801:10::/64

The request to /wp-login.php from 203.0.113.77 is the typical pattern: a scanner borrowing Googlebot’s name.

Common mistakes

Pitfalls with crawler IP lists

The lists are reliable; how they are used often is not.

Copying the list once

Operators add ranges every few weeks. A firewall allowlist built in spring blocks part of Googlebot by autumn. Automate the download.

Treating “not listed” as a bot

Most unlisted addresses are visitors. Use the list to confirm crawlers, not to accuse everyone else.

Checking the CDN’s addresses

If every address in your log belongs to Cloudflare or your load balancer, nothing will match. Log the original client IP first.

Forgetting user-triggered fetchers

ChatGPT-User, Perplexity-User and Google’s user-triggered fetchers act for a person and may ignore robots.txt. They are on separate lists for a reason.

FAQ

Crawler IP range questions

Still have a question?

Chat with the team behind TapCub — we usually reply within a few hours on business days.

Where can I find Googlebot’s IP ranges?

Google publishes them as JSON at developers.google.com/static/crawling/ipranges/: common-crawlers.json (Googlebot and other common crawlers), special-crawlers.json (AdsBot and similar) and three files for user-triggered fetchers and agents. This tool includes all five.

Does OpenAI publish GPTBot’s IP addresses?

Yes: openai.com/gptbot.json for GPTBot, searchbot.json for OAI-SearchBot, chatgpt-user.json for ChatGPT-User and adsbot.json for OAI-AdsBot. OpenAI does not document a reverse DNS check, so the lists are the way to verify its crawlers.

Which crawlers do not publish IP ranges?

Baidu and Yandex document a reverse DNS check instead, and many smaller bots publish nothing at all. Use the fake bot checker for the DNS-verified ones; for the rest, look up the network with the IP lookup tool.

Should I allowlist these ranges in my firewall?

You can, if you refresh them automatically. A stale allowlist blocks new crawler addresses, and an allowlist alone does not stop fakes using other addresses; combine it with user-agent rules.

How often do the lists change?

Each operator updates on its own schedule, and the creation time inside each file shows when it last changed: some files stay the same for years, others change within weeks. This tool shows the date it fetched them (2026-11-17); for a firewall, download the files on a schedule.

Are the addresses I paste uploaded?

No. The lists are downloaded from tapcub.com as a static file and the matching runs in your browser. Nothing you paste is sent or stored.

Changelog

Update history

New features, fixes and data refreshes for this tool, newest first.

  1. · v1.0

    LaunchedFirst release.

Data this tool relies on

  • Official crawler IP ranges (Google, Bing, OpenAI, Anthropic…) — checked 2026-11-17, reviewed every 1 months (next review by 2026-12-17)

Missing something or found an error? Tell us — requests that come up often go into the next update. All tool updates

See your data clearly. Find your growth.

Every click, backed by data. Install one line of code and see your first numbers in a minute.

No credit card · Free plans for analytics and chat · Cookieless analytics