Publisher integration
The console records a verified site; CrawlClick then attaches that host to its own Cloudflare-for-SaaS edge. Ordinary visitors keep the origin path, CrawlClick errors fail open to the real page, and a built-in check compares crawler and human behavior.
The activation
01
Enter the host readers use, such as www.example.com, and a different HTTPS subdomain such as origin.example.com. The origin must serve the same content without looping through the managed edge route. Credentials, paths, private addresses, localhost and metadata endpoints are refused.
02
Add the exact TXT name and value shown in the console. CrawlClick stores its hash and binds a successful public lookup to your organisation. A wrong or missing value cannot verify the site; propagation may take time.
03
Verification creates one-time site activation values. CrawlClick support consumes the site identifier and token inside CrawlClick's Cloudflare account and attaches the verified host as a custom hostname. You do not deploy the billable edge code and never receive the separate platform credential that attests crawler network identity.
04
Point the public host at the CNAME target supplied during activation. Wait until Cloudflare reports the custom hostname and managed certificate active before sending pilot traffic. Human navigation signals remain measurement-only; activation never opts visitors into paid treatment.
05
Choose / or a representative local content path and click Check integration. Ready means the crawler probe was routed, its response is private and no-store, and the ordinary-human probe bypassed crawlclick. Test a second path when route rules vary by section.
06
Complete Stripe Connect Express and return to the console so it can read the real capability status. Start with observation. Enable disclosed insertion only for the agreed network-verified crawler scope after the site owner accepts the measurement and disclosure boundary.
Failure test
Test the route before enabling placements. Record a normal human response, a crawler response, a non-HTML asset, a non-GET request and a deliberately unavailable or misconfigured CrawlClick upstream. Any proxy timeout, authentication failure, stale configuration or serving error must refetch the normal publisher request and return the real page.
A successful proxy response includes explicit private/no-store controls so crawler-specific content cannot enter a shared cache. The human check proves the normal branch does not traverse crawlclick. These are separate assertions; the console reports both.
Credential lifecycle
Use Rotate token, type the exact verified host and ask CrawlClick support to replace the custom-hostname metadata. The old site token stops working. It never exposes or rotates the independent platform credential. Registering the same verified host again is intentionally refused so a routine setup action cannot invalidate a live route.
Acceptance evidence
Keep the verification time, platform edge deployment version, CNAME and managed-certificate status, crawler/human check result, fail-open result, Connect capability status and approved crawler-insertion scope. Do not retain the one-time site token in the acceptance record.
Start in the reviewed flow
Approval creates an organisation boundary; verification creates one site boundary; managed activation creates one custom-hostname boundary. CrawlClick retains control of the platform secret so a revenue recipient cannot attest its own billable network identity.