AI crawlers · DuckDuckGo

DuckAssistBot

DuckAssistBot is a crawler operated by DuckDuckGo, and it fetches your page while a person waits for an answer — the fact that decides everything else on this page.

What does DuckAssistBot actually do?

DuckAssistBot fetches pages for DuckDuckGo's AI answers and does nothing else. The ordinary results do not come from this crawl, so blocking it takes you out of the generated summaries while leaving you in the search results underneath them. That is a far narrower trade than most agents in this registry offer you.

DuckDuckGo does not build its own foundation models - the summaries come from models it licences, reading pages fetched at question time. So this is retrieval rather than corpus-building: your text is read to answer one question, not accumulated into a training set by the company doing the fetching.

Watch the token. DuckDuckGo has an older search crawler under a different name, and a robots rule written for that one does not apply here. If you last touched your DuckDuckGo rules before AI answers existed, whatever you wrote is almost certainly not covering this agent.

Two details from DuckDuckGo's own page make the decision easier than most. A robots change takes effect within 72 hours, so there is a defined window to wait out rather than an open-ended one. And blocking this agent does not affect organic search rankings on DuckDuckGo, which is stated rather than implied - so the trade really is the narrow one it appears to be, with the AI answers given up and the ordinary results kept.

DuckDuckGo also states outright that the content this agent fetches is not used in any way to train AI models, and publishes both a JSON file and an enumerated list of the addresses it crawls from. Between the explicit training disclaimer, the published addresses and the stated propagation window, this is among the better documented entries in the registry - which is not a sentence anyone expected to write about the smallest engine on the list.

CrawlClick files DuckAssistBot as a query time crawl. What the three crawl purposes mean.

DuckDuckGo runs one agent in this registry, which makes DuckAssistBot the whole of the decision. Vendors that split the work across several tokens let a publisher decline one job and keep another; DuckDuckGo publishes no second name here, so there is nothing to separate. Whatever you settle on for DuckAssistBot governs every use DuckDuckGo makes of your pages until another token appears — which is a coarser instrument than it looks, and worth weighing before the query time label decides it for you.

Should I block DuckAssistBot?

An existing DuckDuckBot rule is assumed to cover this agent; the tokens are separate, and only the one naming DuckAssistBot affects AI answers.

To block it anyway, add this to robots.txt (how the rules resolve):

User-agent: DuckAssistBot
Disallow: /

Replace Disallow: / with Allow: / to permit it explicitly. A crawler that ignores robots.txt is not blocked by it either way — that is what verification and edge rules are for.

How do I verify DuckAssistBot is genuine?

DuckDuckGo lets you check a request claiming to be DuckAssistBot against published IP ranges. A user-agent header on its own proves nothing.

Its address list lives at https://duckduckgo.com/duckassistbot.json.

The verification procedure explains why a reverse lookup has to be forward-confirmed, and why a published range list has to be re-fetched rather than pinned.

How do I find DuckAssistBot in my logs?

Match the user-agent against "duckassistbot", case-insensitively. Every DuckAssistBot fetch had somebody waiting on it, so this count maps onto real questions being asked about your pages rather than onto crawl budget.

Crawl-to-referral gap defines the ratio, and the rate-limiting guide covers what to do when the volume itself is the problem.

Registry last reviewed 2026-09-13.