Automated clients
GAIP's user-agents and what they read.
Outbound requests from GAIP identify themselves with one of the tokens below. None sends credentials, logs in or reads beyond public metadata and public endpoints.
| User-agent | Purpose | When it runs | Rate limits | robots.txt |
|---|---|---|---|---|
GAIP-Observatory/1.0 (+https://www.gaipagents.com/bots) | Agent Observatory: reads public Official MCP Registry listings, sends the MCP discovery handshake (initialize, tools/list; never tools/call) to remote endpoints listed there, and reads public A2A agent cards listed by public A2A registries (a2aregistry.org; a2a-registry.org at most weekly, with its operator's written permission). Once per week per host it also reads that host's public discovery documents (OAuth and OpenID metadata, A2A agent card, MCP server card, AI catalogue, x402, llms.txt) and keeps only whether each answered, never the document. | Scheduled. Registry listing at most 20 pages every 2 hours; each target at most once every 24 hours. | At most 2250 requests per hour in total and 9 per target observation; at least 60 seconds between targets on the same host. | Honoured: the GAIP-Observatory group, otherwise the * group. If robots.txt cannot be read (other than 404/410), the host is skipped. Hosts on the opt-out list are skipped. |
GAIP-Conformance-Check/1.0 (+https://www.gaipagents.com/bots) | Check an agent by URL: reads the agent card, MCP server or OpenAPI document a caller submitted and checks it against the published conformance rules. | Only when a caller asks for a check. | One bounded read per submitted URL and discovery path. | Reads only the public URL a caller submitted (and its standard discovery paths). |
GAIP-Citation-Witness/1.0 (+https://www.gaipagents.com/bots) | Citation witness: checks that pages cited by a caller exist and contain the quoted text. | Only when a caller asks, up to 20 citations per call. | At most 10 page reads per host per call and 256 KiB per page; keeps a hash and at most 300 characters of matched excerpt. | Honoured (RFC 9309) for the GAIP-Citation-Witness group, otherwise the * group. |
GAIP-Supplier-Watch/1.0 (+https://www.gaipagents.com/bots) | Supplier Watch: compares published agent cards of a small public cohort over time for material change. | Scheduled, once a day. | At most 20 targets per run and 64 KiB per response; redirects are not followed. | Reads only public agent-card discovery paths (/.well-known/agent-card.json, /.well-known/agent.json). |
GAIP-Watch/1.0 (+https://www.gaipagents.com/bots) | Watch-lists: re-checks an agent card, supplier interface or delivery terms that a caller registered, and sends a signed webhook POST to the caller's own endpoint when it changes. | Scheduled per watch, at most once every 24 hours. | At least 600 seconds between reads of the same host; at most 120 webhook attempts per hour in total. | Reads only the URL the caller registered. |
GAIP-Broker/1.0 (+https://www.gaipagents.com/bots) | Opportunity Broker: reads public agent cards to check which candidates declare a fit for a bounded task. | Only when a caller asks. | At most 256 KiB per document; 5-second timeout. | Reads only the public URL a caller submitted (and its standard discovery paths). |
GAIP-Discovery/1.0 (+https://www.gaipagents.com/bots) | Discovery: reads public agent registry listings to find public agent cards for the aggregate readiness index. | Scheduled. | A bounded number of listing pages per run. | Reads public registry listing pages only. |
GAIP-Adoption-Evidence/1.0 (+https://www.gaipagents.com/bots) | Evidence records: a bounded public reachability read made while recording a caller-requested check. | Only when a caller asks. | One read of at most 32 KiB; public HTTPS on port 443 only. | Reads only the public URL a caller submitted (and its standard discovery paths). |
GAIP-Venture-Scout/1.0 (+https://www.gaipagents.com/bots) | Internal planning: reads GAIP's own public aggregate of requested task kinds. | Scheduled. | One read per run. | Reads GAIP's own endpoint only. |
GAIP-Shadow-Qualification/1.0 (+https://www.gaipagents.com/bots) | Service qualification: a bounded live read of the public services GAIP is listed on (for example registry listings) to record whether they respond as declared. | Scheduled. | One read per listed service per run. | Reads only explicitly approved public service URLs. |
GAIP-Capability-Assurance/1.0 (+https://www.gaipagents.com/bots) | Capability assurance: runs bounded synthetic tasks against GAIP's own public routes. | Scheduled. | A bounded number of synthetic cases per run. | Calls GAIP's own endpoints only. |
GAIP-Verify-Before-Call/2.0 (+https://www.gaipagents.com/bots) | Verify before call: a single optional reachability read of the endpoint a caller named, only when the caller sets live_public_probe. | Only when a caller asks. | One read of at most 32 KiB; public HTTPS on port 443 only; redirects are not followed. | Reads only the public URL a caller submitted (and its standard discovery paths). |
GAIP-Listing-Observer/0.1 (+https://www.gaipagents.com/bots) | Listing Observer (public preview): reads one public product page and the shop platform's own public product document (for example /products/<handle>.js; on a Shopify-hosted shop, only where GAIP's app is installed; otherwise the HTML page only) and records whether price, currency, stock and SKU agree between them. Values only; no page copies, no checkout, no sign-in. Amazon, eBay and Etsy are never read. | When a caller asks (MCP tool gaip_observe_listing) and in manually started accuracy test runs. | At most 3 reads per product page (page, platform document, one second look); GAIP's per-host outbound budget applies. | Honoured (RFC 9309) for the GAIP-Listing-Observer group, otherwise the * group. Hosts on the opt-out list are skipped. |
GAIP-Market-Watch/1.0 (+https://www.gaipagents.com/bots) | Market Watch (internal planning): reads a short, fixed list of public pages (product, pricing and documentation pages, public release feeds and issue pages of open standards, public app-store listings and the Official MCP Registry's public search) to note what changed for GAIP's own planning. Keeps a hash, extracted facts (such as listed prices or a release tag) and at most 300 characters of changed text; nothing it reads is published. | Scheduled, about once a week. | One read per listed page per run (plus robots.txt); at least 8 seconds between reads of the same host; redirects only within the same site, at most two. | Honoured (RFC 9309) for the GAIP-Market-Watch group, otherwise the * group, on every path. Hosts on the opt-out list are skipped. |
GAIP-Permission-Check/1.0 (+https://www.gaipagents.com/bots) | Permission check (gaip_check_permission): when a caller asks "may my agent fetch this URL?", reads that site's /robots.txt, /.well-known/tdmrep.json and /llms.txt once and answers from what they say. Keeps nothing; llms.txt is reported by status, size and hash only. | Only when a caller asks about a URL on the site. | At most three files per question (plus up to five same-site redirects for robots.txt). | /llms.txt is read only when robots.txt lets the GAIP-Permission-Check group (otherwise the * group) read it; robots.txt and /.well-known/ documents are published for automated clients. |
GAIP-Owner-Verification/1.0 (+https://www.gaipagents.com/bots) | Site-owner verification (host timeline): when someone who runs a site asks for GAIP's record about it, reads that site's /.well-known/gaip-verification.txt once to check the token GAIP gave them. Keeps nothing. | Only when the site's owner asks to verify. | One file per request (plus robots.txt); redirects are not followed. | Honoured (RFC 9309) for the GAIP-Owner-Verification group, otherwise the * group. Hosts on the exclusion list are never contacted. |
GAIP-Distribution/1.0 (+https://www.gaipagents.com/bots) | Distribution Agent (internal planning): once a week reads the public pages where GAIP itself is listed (the Official MCP Registry's public search, Glama, Smithery, mcp.so, PyPI, the A2A registries) to check that GAIP's own listings are live and match GAIP's listing kit, and GAIP's own public funnel views. Keeps a status per page and at most 300 characters of a page that reports GAIP as unhealthy; nothing it reads is published and it never posts, submits or contacts anyone. | Scheduled, about once a week. | One read per listed page per run (plus robots.txt); GAIP's per-host outbound budget applies. | Honoured (RFC 9309) for the GAIP-Distribution group, otherwise the * group, on every path. Hosts on the opt-out list are skipped. |
GAIP-Shopify-App-Shadow/1.0 (+https://www.gaipagents.com/bots) | Shopify App Shadow (internal planning): once a week reads GAIP's own Shopify app pages (its public page and its daily health answer) and GAIP's own UCP profile, to check the app works for merchants. It reads no third-party site; nothing it reads is published and it never posts, submits or contacts anyone. | Scheduled, about once a week. | Three reads of GAIP's own pages per run (plus robots.txt); GAIP's per-host outbound budget applies. | Honoured (RFC 9309) for the GAIP-Shopify-App-Shadow group, otherwise the * group, on every path. |
GAIP-Recall-Check/1.0 (+https://www.gaipagents.com/bots) | Recall check (Second Look by GAIP and gaip_recall_check): when someone checks a product, searches two public recall lists for its brand, model and barcode: the GOV.UK search API (product safety alerts, reports and recalls) and the US CPSC recalls API. Keeps only a found notice's title, date and link. | Only when someone checks a product; answers are reused for six hours. | At most one search on each list per product checked; GAIP's per-host outbound budget applies. | Honoured (RFC 9309) for the GAIP-Recall-Check group, otherwise the * group. It reads only the two public APIs named here. |
GAIP-Company-Register/1.0 (+https://www.gaipagents.com/bots) | Company register lookup (Second Look by GAIP, who is behind a shop): when a shop's own pages show a UK company number, looks that number up on the Companies House public register API. It reads no shop and keeps only the register's name, status and dates for that company. | Only when someone checks a product whose shop shows a company number; answers are reused for a day. | At most one lookup per company number checked; Companies House allows GAIP 600 requests per five minutes. | It reads only api.company-information.service.gov.uk. |
GAIP-Routing-Table/1.0 (+https://www.gaipagents.com/bots) | GAIP automated client. | See purpose. | Bounded by GAIP's outbound request limits. | Reads only public documents. To ask GAIP to stop, use the corrections and opt-out page. |
GAIP-Served/1.0 (+https://www.gaipagents.com/bots) | GAIP Served (witness_serve): when a sender asks GAIP to deliver a cancellation, complaint, withdrawal, termination or dispute notice to an agent endpoint, and only where that endpoint's own preference record allows service notices through GAIP, reads the endpoint host's agent card once and sends the notice to the endpoint itself (an A2A message, an MCP tools/call or a JSON POST), once more ten minutes later if the first got no 2xx answer. | Only when a sender serves a notice on that endpoint. | Two requests per notice (card and delivery), plus one retry; one notice per sender per endpoint per day. | Not read for the delivery: the endpoint opted in through its own published preference record; hosts on the exclusion list are never contacted. |
GAIP-Status-Check/1.0 (+https://www.gaipagents.com/bots) | GAIP's status page (/status): once an hour GAIP's Railway runtime reads GAIP's own www.gaipagents.com/v1/free/collection-status to record whether the public site answered. | Hourly. | One request an hour (plus robots.txt). | Reads only GAIP's own site. |
GAIP-Compliance-Record/1.0 (+https://www.gaipagents.com/bots) | Compliance record (GAIP's Shopify app): reads once a day each product page a merchant chose and consented to in the app, and keeps the price, currency, availability, any struck-through price and any countdown the page showed (values and a hash of the countdown text only), and whether the page could be read. Reads only the consenting shop's own pages; never a cart, checkout or account page; Amazon, eBay and Etsy never. | Scheduled, once a day per consented product. | One read per product per UTC day; at most 20 reads per 15-minute tick and 4,000 a day across every shop; GAIP's per-host outbound budget applies. | Honoured (RFC 9309) for the GAIP-Compliance-Record group, otherwise the * group. Hosts on the opt-out list are skipped even where the merchant consented. |
GAIP-Recall-Watch/1.0 (+https://www.gaipagents.com/bots) | Recall watch (the act log, gaip_recall_watch): once a day reads two public recall lists, the GOV.UK search API (product safety alerts, reports and recalls) and the US CPSC recalls API, for notices published since the previous read, and matches them against the product identifiers agents registered (kept as keyed hashes). Keeps only a matched notice's title, date and link. It reads no shop and no other site. | Scheduled, once a day. | At most eight requests a day across both lists (a few pages of each); GAIP's per-host outbound budget applies. | Honoured (RFC 9309) for the GAIP-Recall-Watch group, otherwise the * group. It reads only the two public APIs named here. |
Opt out or correct
To stop scheduled reads of your hosts or to correct an observation, use corrections and opt-out or the machine route POST /v1/free/observatory/corrections. To block one token, add a robots.txt group for it, for example User-agent: GAIP-Observatory followed by Disallow: /.
Machine-readable: /bots.json · Methodology
Signed requests (Web Bot Auth)
GAIP's requests are not signed at present: no signing key is set. Last recorded by the runtime: 2026-10-17T22:42:03Z.
When signing is on, each request carries Signature-Agent, Signature-Input and Signature headers (HTTP Message Signatures, RFC 9421, Ed25519) covering the host and the Signature-Agent value, with a short expiry. The public key is at /.well-known/http-message-signatures-directory; a site can use it to tell GAIP's requests from someone else using GAIP's user-agent.