AI crawlers · OpenAI

ChatGPT-User

ChatGPT-User is a crawler operated by OpenAI, and it fetches your page while a person waits for an answer — the fact that decides everything else on this page.

What does ChatGPT-User actually do?

ChatGPT-User is not a crawler in the traversal sense. It opens the specific URL a conversation needs - a link somebody pasted, a page the model chose to read, a URL handed to it by a connector or a custom GPT - and then stops. No link discovery, no sitemap, no scheduled return. One request, one page, usually while a person watches a spinner.

That makes it the only OpenAI agent where your response time is part of the product. A slow origin, a bot-challenge interstitial or a consent wall does not get retried politely; the answer gets assembled from whatever else is available. Publishers who put a JavaScript challenge in front of this agent have opted out of being read at the moment of the question, usually without meaning to.

It is also the OpenAI agent most worth checking properly, because a header claiming a person asked for the page is a convenient story for a scraper to tell. This is one of the few agents in the registry where the vendor supports signed requests, which is better evidence than an address file you have to keep refreshing.

OpenAI states in its own documentation that because these actions are initiated by a user, robots.txt rules may not apply to this agent. That single sentence decides what a disallow line can achieve here: it is a request rather than a control. If you genuinely want these fetches stopped, the rule has to live at your edge, where you can enforce it, rather than in a file the vendor has said in advance may not bind it.

CrawlClick files ChatGPT-User as a query time crawl. What the three crawl purposes mean.

A robots rule that names OpenAI rather than ChatGPT-User also lands on OAI-SearchBot (search index), GPTBot (training), which does a different job. Blocking the wrong one of a vendor's agents is the commonest mistake in this whole subject.

Should I block ChatGPT-User?

Seeing ChatGPT-User in the logs is read as proof the site is being harvested for training, when this agent fetches one page for one live conversation and OpenAI runs training under a separate name.

To block it anyway, add this to robots.txt (how the rules resolve):

User-agent: ChatGPT-User
Disallow: /

Replace Disallow: / with Allow: / to permit it explicitly. A crawler that ignores robots.txt is not blocked by it either way — that is what verification and edge rules are for.

How do I verify ChatGPT-User is genuine?

OpenAI lets you check a request claiming to be ChatGPT-User against signed request (Web Bot Auth), published IP ranges. A user-agent header on its own proves nothing.

Its address list lives at https://openai.com/chatgpt-user.json.

The verification procedure explains why a reverse lookup has to be forward-confirmed, and why a published range list has to be re-fetched rather than pinned.

How do I find ChatGPT-User in my logs?

Match the user-agent against "chatgpt-user", case-insensitively. Every ChatGPT-User fetch had somebody waiting on it, so this count maps onto real questions being asked about your pages rather than onto crawl budget.

Crawl-to-referral gap defines the ratio, and the rate-limiting guide covers what to do when the volume itself is the problem.

Registry last reviewed 2026-09-13.