Robots.txt Checker
& AI Bot Analyzer
Check if AI bots like GPTBot, ClaudeBot, and PerplexityBot can crawl your website. See which crawlers are blocked and get recommendations.
Enter a domain above to check its robots.txt and see which AI bots can crawl it.
How to Use This Tool
Enter a Domain
Paste any domain you want to analyze.
See Bot Access
View which AI and search engine bots can crawl the site.
Fix Issues
Update your robots.txt to allow AI bots if needed.
Frequently Asked Questions
Everything you need to know about RankGap and improving your visibility in both Google and AI search.
Robots.txt is a text file that tells web crawlers which pages they can or cannot access on your website. It's located at the root of your domain (example.com/robots.txt).
AI platforms like ChatGPT, Perplexity, and Claude use bots to learn about websites. If you block them, your site won't be cited or recommended in AI-generated responses.
GPTBot is OpenAI's web crawler that collects data for training ChatGPT and related products. Many sites block it due to copyright concerns, but this may hurt your visibility in AI platforms.
Yes, this robots.txt checker is completely free. For a comprehensive AI visibility audit including 12 metrics, check out RankGap's full dashboard.
Related SEO Tools
Continue improving your site with these free tools
Sitemap Checker
Validate your XML sitemap for free. Check URL count, lastmod dates, and find issues that could affect your SEO.
Schema Validator
Validate your JSON-LD structured data for free. Check for required fields and get recommendations to improve your schema markup.
Security Checker
Check HTTPS, security headers, and mixed content issues. Analyze your website's security configuration.