GPTBot Blocked? Here's What to Do (and What NOT to Do)
If your robots.txt blocks GPTBot, you've opted out of OpenAI model training — which may be intentional. The real danger is accidentally blocking the wrong crawler along the way. Here's how to fix it the right way.
"Should I block GPTBot?" is the wrong question. The right question is: which OpenAI crawlers do I want — and which don't I?
What blocking GPTBot actually does
GPTBot is OpenAI's training crawler. Blocking it (User-agent: GPTBot Disallow: /) opts your content out of OpenAI's model training. It does not remove you from ChatGPT answers — that's governed by OAI-SearchBot, a completely separate crawler.
The mistake to avoid
Blocking GPTBot "to protect content" is fine. But blanket blocking often sweeps in the retrieval crawlers too:
# ❌ What many sites end up with by accident:
User-agent: *
Disallow: / # blocks EVERYONE including OAI-SearchBot
# ❌ Or this common trap:
User-agent: GPTBot
User-agent: OAI-SearchBot # unintentionally treating them the same
Disallow: /The result: you opted out of training and removed yourself from ChatGPT answers. That's the most expensive mistake in AI SEO — you gave up nothing, and lost citations.
What to do
Decide per crawler, not as one blob:
# Block training, keep citations (recommended default for most sites):
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /If you want your content in training data (many sites do — it's free distribution), just allow GPTBot. The point: make each decision explicit.
Verify after editing
Edit robots.txt, then re-run a check to confirm: GPTBot blocked (if intended), OAI-SearchBot and ChatGPT-User allowed. Run the free 24-crawler check — it shows the exact rule behind each result.