See which AI crawlers can read your site — GPTBot, ClaudeBot, PerplexityBot & 20 more. Curated, operator-sourced bot list + a zero-dep CLI and GitHub Action to audit robots.txt, test reachability, read access logs, and gate it all in CI.

Complete database of AI search engine crawler user-agents (GPTBot, PerplexityBot, ClaudeBot) with robots.txt configuration examples

Agent-native multi-bot web crawler simulator. See your site through the eyes of Googlebot, GPTBot, ClaudeBot, and PerplexityBot.

Validates whether GPTBot, ClaudeBot, PerplexityBot and other LLM crawlers are allowed by a robots.txt file.

User-agent simulator for major LLM crawlers: measures response code, content length, and JS-dependency

Visual robots.txt manager for the AI era. Block or allow GPTBot, ClaudeBot, PerplexityBot, and 20+ AI crawlers with one click.

The best AEO/GEO skill for Claude Code. Audit, fix, and monitor your website's visibility across ChatGPT, Claude, Perplexity, Gemini, and Google AI Overviews. 100+ optimization techniques, 33 evidence collectors, 4-vector GEO Score. Built on Princeton KDD 2024 paper. Free, MIT.

Free no-tracking browser utility for drafting and auditing AI crawler robots.txt rules.