Meta AI search index; blocking removes you from Meta AI answers.
Almost never block
meta-webindexer builds and refreshes the index behind Meta's AI answers. Blocking it is the most expensive mistake in AI SEO: you disappear from Meta citations while your training decision stays completely separate.
If you need to protect specific sections (pricing, docs), use path-scoped rules instead of a blanket Disallow: /.
User-agent: meta-webindexer Allow: /
User-agent: meta-webindexer Disallow: /
Community source Community-maintained list — last reviewed 2026-09-12
https://github.com/ai-robots-txt/ai.robots.txt
We have not verified this entry against the operator's own documentation. Treat the description as indicative, and check the operator's site before relying on it.
"Should you block it?" below is our own interpretation, not the operator's recommendation.
See whether meta-webindexer — and 39 other AI crawlers — can reach your site today.