GPTBot
OpenAI's web crawler — used to index content for ChatGPT's training data and retrieval-augmented generation (RAG) in ChatGPT Search.
KEY FACTS
- —User-agent string: GPTBot/1.1
- —Respects robots.txt — block with: User-agent: GPTBot / Disallow: /
- —Crawls for both training data and live SearchGPT retrieval
- —Visits most pages every 2–4 weeks based on content freshness signals
- —Blocking GPTBot makes your site invisible to all ChatGPT-powered recommendations
- —OpenAI publishes the full GPTBot IP range for firewall whitelisting
FREQUENTLY ASKED
What is GPTBot?
GPTBot is OpenAI's web crawler. It indexes websites to improve ChatGPT's knowledge and power ChatGPT Search recommendations. If GPTBot is blocked, your site doesn't appear in ChatGPT answers.
Should I block or allow GPTBot?
Allow it if you want to appear in ChatGPT recommendations. Block it only if you have content you don't want used in AI training. Most businesses benefit from allowing GPTBot.
How do I allow GPTBot in robots.txt?
Add these two lines to your robots.txt: User-agent: GPTBot Allow: /
RELATED TERMS
FREE TOOL
Check your AI visibility score — free in 60 seconds.
See exactly how you score on gptbot and four other AI visibility factors. Average score is 23/100.
Score My Site →ZYGUR TECHNOLOGIES CORP. · AI SEARCH OPTIMIZATION KNOWLEDGE BASE · ZYGUR.COM/KB