Is my site blocked by ChatGPT?
Check whether ChatGPT, Perplexity, Claude, Google and Bing can crawl a page. We read robots.txt for 15 AI bots and send live test requests with real AI-bot user agents to catch firewall and CDN blocks.
How does the AI crawler check work?
We fetch your robots.txt and apply the same matching rules crawlers use (RFC 9309: the most specific user-agent group wins, then the longest matching path). Then we request your page as a normal browser and as four AI bots and compare the responses. A 403, a challenge page or much less content means a possible firewall block.
Which AI bots should I allow?
- Search and answer bots (OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot, Googlebot, Bingbot): allow these if you want to appear in AI answers.
- Training bots (GPTBot, ClaudeBot, CCBot, Google-Extended): your choice. Blocking them doesn't remove you from AI search.
What is the difference between OAI-SearchBot and GPTBot?
OAI-SearchBot indexes pages for ChatGPT search answers. GPTBot collects data to train OpenAI models. You can block GPTBot and still appear in ChatGPT search.
Why does it say "possible block"?
Some firewalls verify real crawlers by IP address. Our test request comes from our servers, so it can be blocked while the real crawler gets through. Confirm by searching your firewall or server logs for the bot's user agent.
Last updated:
Crawler access is only 22 of 100 points
The full audit also checks JavaScript rendering, schema, structure and quotability, with copy-paste fixes.
Run the full audit