Tag
llms.txt
2 articles tagged “llms.txt”. Tags cut across categories, so you may find analytics, chat and product notes side by side.
Tagged llms.txt
2 articles

AI SearchUpdated NovSep 17, 2026 · 8 min read
llms.txt: what to write, an example, and how to check whether anyone reads it
What llms.txt is and is not, the structure the proposal defines, a real example, how it should agree with robots.txt, and how to check in your server logs whether AI agents fetch it.
Read more
AI SearchUpdated NovMar 26, 2026 · 9 min read
How to block AI crawlers: user agents, robots.txt rules and how to check them
GPTBot, ClaudeBot, Google-Extended and the rest do different jobs. Which AI crawlers to block, the robots.txt rules to use, what to enforce at the server, and how to check it worked.
Read more