AIAuditFix
Tools
Planned
Glossary
Leaderboard
← All tools
robots.txt Generator for AI
Decide each AI crawler's access and your content-usage signals, then copy the result.
AI crawler rules
GPTBot
Not addressed
Allow
Block
ClaudeBot
Not addressed
Allow
Block
anthropic-ai
Not addressed
Allow
Block
PerplexityBot
Not addressed
Allow
Block
CCBot
Not addressed
Allow
Block
Applebot-Extended
Not addressed
Allow
Block
Google-Extended
Not addressed
Allow
Block
Bytespider
Not addressed
Allow
Block
Content-Signal preferences
search
ai-input
ai-train
Sitemap URL
(optional)
Generate