AI crawlers · Google

GoogleOther

GoogleOther is a crawler operated by Google, and it builds the index an assistant answers from — the fact that decides everything else on this page.

What does GoogleOther actually do?

GoogleOther is what Google's own teams use when they need pages for something other than Search: internal research and development, one-off corpora, product experiments. Google's documentation states plainly that it is separate from Googlebot and that blocking it does not affect Search indexing or ranking. Few crawler decisions come with that clear an answer.

Google introduced it so these fetches would stop consuming Googlebot's crawl capacity, which is also why blocking it frees nothing up for Search - the two were deliberately separated. What is left to decide is not a traffic question at all. It is a judgement about what you are content for Google to hold a copy of outside Search.

There are image and video variants under the same family name, and Google lists the base token as applying to them too, so one robots rule covers all three. Your log parser inherits none of that, though: match on the base string only and the variants either vanish from your counts or merge into them, depending on how the match is written.

This is also the token people reach for when they meant a different one. Google-Extended is the control that governs whether already-crawled material may train Gemini; it is a robots token with no fetcher behind it, so no request ever carries that name and no amount of log analysis will show it to you. A publisher who wanted out of Gemini training and wrote a rule against this agent instead has declined a research crawl and left the training permission exactly as it was.

CrawlClick files GoogleOther as a search index crawl. What the three crawl purposes mean.

A robots rule that names Google rather than GoogleOther also lands on Google-CloudVertexBot (query time), Googlebot (search index), which does a different job. Blocking the wrong one of a vendor's agents is the commonest mistake in this whole subject.

Should I block GoogleOther?

It is assumed to feed Search, so publishers who want it gone hesitate to block it - Google states it has no effect on Search indexing or ranking.

To block it anyway, add this to robots.txt (how the rules resolve):

User-agent: GoogleOther
Disallow: /

Replace Disallow: / with Allow: / to permit it explicitly. A crawler that ignores robots.txt is not blocked by it either way — that is what verification and edge rules are for.

How do I verify GoogleOther is genuine?

Google lets you check a request claiming to be GoogleOther against reverse DNS ending in .google.com or .googlebot.com, published IP ranges. A user-agent header on its own proves nothing.

Its address list lives at https://developers.google.com/static/crawling/ipranges/special-crawlers.json, and its reverse-DNS names end in .google.com or .googlebot.com.

The verification procedure explains why a reverse lookup has to be forward-confirmed, and why a published range list has to be re-fetched rather than pinned.

How do I find GoogleOther in my logs?

Match the user-agent against "googleother", case-insensitively. Set the GoogleOther fetch count beside the visits Google actually returns; that ratio is your crawl-to-referral gap for this engine.

Crawl-to-referral gap defines the ratio, and the rate-limiting guide covers what to do when the volume itself is the problem.

Registry last reviewed 2026-09-13.