AI crawlers · Amazon

Amazonbot

Amazonbot is a crawler operated by Amazon, and it builds the index an assistant answers from — the fact that decides everything else on this page.

What does Amazonbot actually do?

Amazonbot serves Alexa and Amazon's own answer surfaces rather than a public search engine you can check yourself against. That is the awkward part of the decision: there is no results page showing what allowing it bought you, so the crawl-against-referrals comparison that works for search engines tells you very little here.

Amazon states plainly that these agents do not support the crawl-delay directive, so a volume problem cannot be solved in robots.txt here at all. The same page adds that the bot fetches a host-level robots.txt or uses a cached copy from the last thirty days, which means a rule you add today may go on being ignored for a month before it starts to bite.

The verification route Amazon actually publishes is a list of addresses, not a hostname pattern: there is a JSON file of individual Amazonbot IPs and no documented reverse-DNS procedure to fall back on. Treat a hostname ending in amazonaws.com as evidence against rather than for, because that is a machine rented from Amazon rather than anything Amazon operates, and the two get confused constantly.

Read Amazon's own description of the purpose before filing this one as a search crawl. The page says the fetches improve Amazon's products and services and may train Amazon AI models, which puts it on both sides of a line most vendors keep separate with two tokens. There is no second name to allow or deny here, so the training question and the visibility question arrive as a single decision whether you wanted them to or not.

CrawlClick files Amazonbot as a search index crawl. What the three crawl purposes mean.

Amazon runs one agent in this registry, which makes Amazonbot the whole of the decision. Vendors that split the work across several tokens let a publisher decline one job and keep another; Amazon publishes no second name here, so there is nothing to separate. Whatever you settle on for Amazonbot governs every use Amazon makes of your pages until another token appears — which is a coarser instrument than it looks, and worth weighing before the search index label decides it for you.

Should I block Amazonbot?

Traffic from AWS address space gets filed as Amazonbot, when most of it is unrelated scrapers renting Amazon servers rather than anything Amazon operates.

To block it anyway, add this to robots.txt (how the rules resolve):

User-agent: Amazonbot
Disallow: /

Replace Disallow: / with Allow: / to permit it explicitly. A crawler that ignores robots.txt is not blocked by it either way — that is what verification and edge rules are for.

How do I verify Amazonbot is genuine?

Amazon lets you check a request claiming to be Amazonbot against published IP ranges. A user-agent header on its own proves nothing.

Its address list lives at https://developer.amazon.com/amazonbot/ip-addresses/.

The verification procedure explains why a reverse lookup has to be forward-confirmed, and why a published range list has to be re-fetched rather than pinned.

How do I find Amazonbot in my logs?

Match the user-agent against "amazonbot", case-insensitively. Set the Amazonbot fetch count beside the visits Amazon actually returns; that ratio is your crawl-to-referral gap for this engine.

Crawl-to-referral gap defines the ratio, and the rate-limiting guide covers what to do when the volume itself is the problem.

Registry last reviewed 2026-09-13.