ChatGPT, Perplexity & Claude only cite what their crawlers can reach. Check your robots.txt against 24 major AI crawlers and see who can — and who can't — see you.
Search is shifting from link networks to answer networks. If AI crawlers can't reach you, you don't exist in AI search.
58.5% of Google searches are now zero-click. 42% of buyers already use AI search in their evaluation.
Blocking a retrieval crawler while trying to block training silently removes you from AI answers.
150+ AI user agents and growing. A robots.txt that's fine today can break tomorrow.
One check, three things revealed.
Any public website — yours, a client's, a competitor's.
Fetch and parse it against the full AI crawler roster.
Per-crawler access, the rule that caused it, and fix suggestions.
Training, retrieval and user-triggered — we tell them apart so you don't make the expensive mistake.
Run a check, then subscribe to daily monitoring — we'll email you the moment your AI visibility changes.
Check your site nowPractical guides on AI search, AEO and the crawlers that decide who gets cited.
Everything you need to know before you check your site.
Yes. Single checks are completely free with no signup. You only need an account if you want continuous monitoring and email alerts.
We check 24 major AI crawlers across three types: training (GPTBot, ClaudeBot, Google-Extended, CCBot, Bytespider…), retrieval (OAI-SearchBot, Claude-SearchBot, PerplexityBot, Applebot…) and user-triggered (ChatGPT-User, Claude-User, Perplexity-User).
GPTBot is OpenAI's training crawler — blocking it opts you out of model training. OAI-SearchBot is the retrieval crawler that gets you cited in ChatGPT answers — blocking it removes you from ChatGPT results without affecting training. Blocking the wrong one is the most common (and most expensive) mistake.
robots.txt is a cooperative protocol — well-behaved crawlers (most major AI companies) respect it. Some crawlers (e.g. Bytespider) are known to ignore it, which is why we also flag which crawlers may ignore robots.txt.
Our check shows the exact robots.txt rule blocking each crawler, plus a suggested fix (usually an explicit Allow for the retrieval crawler you need). Edit your robots.txt, then re-run the check to confirm the fix took effect.
Once subscribed, we check your site daily, keep a 30-day history, and email you the moment any AI crawler's access changes — so an accidental block never silently removes you from AI answers.
Anonymous free checks are processed and shown instantly; we do not store those results. URLs are stored only when you subscribe to monitoring, so we can track your site's history over time.
Yes. The checker works on any public website. Many SEO professionals use it to audit client sites and compare their AI visibility against competitors.