AI crawlers · OpenAI

GPTBot

GPTBot is a crawler operated by OpenAI, and it collects text to train a model — the fact that decides everything else on this page.

What does GPTBot actually do?

GPTBot is the block most publishers actually want, and the one with the fewest side effects. Nothing in ChatGPT's answering path depends on it. Allowing or denying it changes what a future model has read, not whether ChatGPT can find you today, and the two OpenAI agents that do affect that are unaffected by the rule.

It is probably the most forged user-agent in this registry, because it is the name everyone knows. Anything can put GPTBot in a header. OpenAI publishes the address file precisely so you do not have to take the header at its word, and traffic claiming this name from outside those ranges is somebody else's scraper wearing a familiar coat.

Nothing about a training fetch is time-sensitive. Nobody is waiting on it, so throttling it hard, deprioritising it at the edge or serving it entirely from cache costs you nothing. That makes it a sensible place to test crawler rules before you point them at agents that answer live questions and fail visibly when you get it wrong.

Rules here do not take effect the moment you save them. OpenAI documents roughly a day between a robots.txt change and its systems acting on it, which is long enough that a publisher who adds a disallow and checks the logs that afternoon concludes it was ignored. Wait out the propagation window before escalating to an edge rule, and make the two changes separately so you can tell which one did the work.

CrawlClick files GPTBot as a training crawl. What the three crawl purposes mean.

A robots rule that names OpenAI rather than GPTBot also lands on ChatGPT-User (query time), OAI-SearchBot (search index), which does a different job. Blocking the wrong one of a vendor's agents is the commonest mistake in this whole subject.

Should I block GPTBot?

GPTBot gets blocked as though it were the OpenAI crawler, leaving OAI-SearchBot and ChatGPT-User reading the site exactly as before.

To block it, add this to robots.txt (how the rules resolve):

User-agent: GPTBot
Disallow: /

Replace Disallow: / with Allow: / to permit it explicitly. A crawler that ignores robots.txt is not blocked by it either way — that is what verification and edge rules are for.

How do I verify GPTBot is genuine?

OpenAI lets you check a request claiming to be GPTBot against published IP ranges. A user-agent header on its own proves nothing.

Its address list lives at https://openai.com/gptbot.json.

The verification procedure explains why a reverse lookup has to be forward-confirmed, and why a published range list has to be re-fetched rather than pinned.

How do I find GPTBot in my logs?

Match the user-agent against "gptbot", case-insensitively. Nothing comes back from a training fetch, so keep GPTBot out of any count you use to judge OpenAI as a source of readers.

Crawl-to-referral gap defines the ratio, and the rate-limiting guide covers what to do when the volume itself is the problem.

Registry last reviewed 2026-09-13.