CrawlClick

GPTBot

OpenAI's training crawler, which collects web content used to improve its models rather than to answer any specific user question.

Also called: OpenAI crawler

Whatever GPTBot reads today feeds a model released months from now. No answer changes because of the visit, and no reader arrives because of it.

Is GPTBot the same as ChatGPT-User?

No, and the difference decides what a visit is worth. ChatGPT-User fetches a page because somebody has just asked something and is sitting there waiting. OAI-SearchBot maintains the index behind ChatGPT Search. Three agents, three jobs, one vendor.

How do I block it precisely?

Two lines in robots.txt at your site root: User-agent: GPTBot followed by Disallow: /. Naming the agent that way leaves the other two untouched, so you stay present in answers while dropping out of the training corpus.

Does blocking undo what was already taken?

No. The exclusion protocol governs future requests only, and text gathered before you added the rule sits inside whatever was trained on it. Publishers expecting a retroactive effect are usually disappointed.

Can I close off part of the site instead?

Yes. Disallow: takes a path, so an archive can be shut while the reference pages you want credited in answers stay open. Blanket rules are a habit rather than a requirement.

How is a claimed GPTBot verified?

By network address, not by header. OpenAI publishes the ranges its crawler uses as a JSON file, and a request from outside those ranges wearing the same user-agent string is a scraper in costume.

Is GPTBot traffic monetizable?

Never. A disclosed sponsored passage is written to be read by a person, and burying one in a training corpus puts the claim somewhere the label cannot follow it.

What else should I read?