What is GPTBot?
GPTBot is OpenAI's crawler for collecting content that may be used to train its models. Blocking it in robots.txt opts your site out of that training; it does not affect ChatGPT search, which uses OAI-SearchBot.
User-agent: GPTBot
Disallow: /
A third OpenAI agent, ChatGPT-User, visits a page when a person asks ChatGPT to open it. OpenAI says robots.txt rules may not apply to it, because a person triggers the visit.
Updated
Guides that cover this
How to allow GPTBot (and ChatGPT search) in robots.txt
2 min readAdd a User-agent: GPTBot group with Allow: / to robots.txt, and make sure no broader rule blocks it. If you want to appear in ChatGPT's search answers, allow OAI-SearchBot too. It's a separate crawler, and it's the one that decides whether ChatGPT search can show your pages.
Block AI training but stay in AI search answers
2 min readDisallow GPTBot, ClaudeBot, Google-Extended, Applebot-Extended and CCBot in robots.txt, and keep OAI-SearchBot, Claude-SearchBot, Claude-User, PerplexityBot and Googlebot allowed. The full file is below.