TapCub is live — analytics, insights and live chat, free on one platform
Tag

AI search

10 articles tagged “AI search”. Tags cut across categories, so you may find analytics, chat and product notes side by side.

Tagged AI search

10 articles

Sample data: PerplexityBot read 8,000 pages and sent back 120 visitors (1.5%, ≈67 reads per visitor); the training crawler read 6,000 pages and sent back ≈0 (0%). In TapCub the ledger shows the same row as about 67 reads : 1 visitor, and AI visibility shows Crawl→visit 1.5%.
AI SearchNov 16, 2026 · 10 min read

PerplexityBot: What It Reads, What It Sends Back, and Whether to Allow It

PerplexityBot reads pages so Perplexity answers can cite your site; Perplexity-User fetches on user request. What each does, robots.txt rules, IP verification, and a reads-vs-visitors rule for allowing or blocking.

Read more
Sample data: server records. Applebot verified by reverse DNS to applebot.apple.com; Extended never crawls; the wrong rule leaves only robots.txt fetches.
AI SearchNov 16, 2026 · 10 min read

Applebot and Applebot-Extended: Two Agents and How to Control Each One

Applebot crawls for Spotlight, Siri and Safari search; Applebot-Extended is only a robots.txt switch for AI training. What each does, three rule choices, DNS and IP verification, and a worked example of what the wrong block costs.

Read more
Zero-click searches: what the data shows and how to adapt
AI SearchNov 6, 2026 · 5 min read

Zero-click searches: what the data shows and how to adapt

About 60% of Google searches now end without a click. What zero-click searches are, which queries are most affected, a worked before/after where impressions rose 4% and clicks fell 20%, and seven ways to adapt.

Read more
Web crawler: how it works, types of crawlers and how to verify them
AI SearchNov 6, 2026 · 6 min read

Web crawler: how it works, types of crawlers and how to verify them

What a web crawler is, the six steps every crawler follows, search vs AI training vs AI search crawlers, a worked week of server logs with spoofed Googlebots, and how to verify and control crawlers.

Read more
AI visibility: how to measure and improve it in AI search
AI SearchNov 6, 2026 · 7 min read

AI visibility: how to measure and improve it in AI search

AI visibility is how often AI answers read, cite and send people to your site. Three layers to measure, four metrics, a worked month of AI traffic by operator, and seven ways to improve it.

Read more
JavaScript SEO: how Google and AI crawlers handle JS, and how to fix it
AI SearchNov 5, 2026 · 6 min read

JavaScript SEO: how Google and AI crawlers handle JS, and how to fix it

JavaScript SEO makes sure crawlers can see content that scripts produce. How Google crawls, renders and indexes JS, why most AI crawlers see an empty shell, rendering strategies, common problems and a catalog audit that went from 53% to 94% indexed.

Read more
Demo workspace with log import. After the rule, GPTBot only fetched robots.txt.
AI SearchNov 3, 2026 · 6 min read

GPTBot: OpenAI's Four Crawlers and How to Control Each One

GPTBot is OpenAI's training crawler, one of four OpenAI agents with independent robots.txt settings. What each does, what opting out changes, rules for common goals, and how to verify and check them in your own data.

Read more
Demo workspace. Requests claiming ClaudeBot from addresses outside Anthropic's list are not verified.
AI SearchNov 3, 2026 · 6 min read

ClaudeBot: What It Crawls, How to Block It and How to Verify It

ClaudeBot is one of three Anthropic web agents. What each one does, what blocking it changes, robots.txt rules for common goals, how to verify requests against Anthropic's published IP list, and how to see ClaudeBot in your own data.

Read more
Sample from the demo project. The compliance page fetches both files and compares robots.txt rules with real fetches; llms.txt fetch counts come from your server logs.
AI SearchUpdated NovSep 17, 2026 · 8 min read

llms.txt: what to write, an example, and how to check whether anyone reads it

What llms.txt is and is not, the structure the proposal defines, a real example, how it should agree with robots.txt, and how to check in your server logs whether AI agents fetch it.

Read more
Sample workspace. Highlighted row: the operator that sends the most people per read.
AI SearchUpdated NovMar 26, 2026 · 9 min read

How to block AI crawlers: user agents, robots.txt rules and how to check them

GPTBot, ClaudeBot, Google-Extended and the rest do different jobs. Which AI crawlers to block, the robots.txt rules to use, what to enforce at the server, and how to check it worked.

Read more
Newsletter

One email a month. Only when there is something worth reading.

New articles, what shipped in the product, and the occasional checklist you can forward to your team.

  • Articles first. Every issue leads with the newest long-form note, summarised in three lines.
  • Product changes that matter. New reports, new SDKs, changed defaults. No marketing filler.
  • Unsubscribe in one click. The link is in every email and it works immediately.

Sent roughly once a month. We never share your address.

Topics you care about

By subscribing you agree to receive the TapCub newsletter. See the privacy policy.

You are on the list.

The next issue goes out at the start of the month. Until then, every article is here on the blog.

See your data clearly. Find your growth.

Every click, backed by data. Install one line of code and see your first numbers in a minute.

No credit card · Free plans for analytics and chat · Cookieless analytics