What is Claude-SearchBot?

In short

Claude-SearchBot is Anthropic's crawler that indexes web pages to improve Claude's search results. Blocking it can reduce how often your content appears when Claude searches the web.

If you want Claude to find and cite your pages, keep Claude-SearchBot allowed even if you block ClaudeBot for training.

User-agent: Claude-SearchBot
Allow: /

Updated October 30, 2026

Guides that cover this

AI crawlers list: every bot name for robots.txt
3 min read
To be cited in AI answers, allow the search crawlers: OAI-SearchBot (ChatGPT), Claude-SearchBot (Claude), PerplexityBot (Perplexity) and Googlebot (AI Overviews and AI Mode). GPTBot, ClaudeBot, Google-Extended and CCBot are about model training, and blocking them doesn't remove you from AI search.

Related terms

ClaudeBot OAI-SearchBot PerplexityBot

Check your own siteSee which AI crawlers your robots.txt lets in and what stops assistants quoting you.Run the free check

What is Claude-SearchBot?

Guides that cover this

AI crawlers list: every bot name for robots.txt

Related terms