Globetrotters AI Presence — WordPress plugin

Serves your Official AI Presence at a WordPress site's apex (AI discovery files + server-rendered JSON-LD), pulled automatically from Globetrotters. No filesystem access required — the plugin hooks WordPress's own router. It also reports agent traffic to those paths back to Globetrotters (see Agent traffic reporting) and updates itself from a Globetrotters-hosted channel (see Versions and updates).

User-facing docs live in readme.txt (WordPress.org format). This README is for development.

• Plugin page and download: https://globetrotters.ai/wp-plugin
• Not on WordPress? The same behaviour for Symfony apps: globetrotters-ai/gt-symfony-bundle (composer require globetrotters-ai/symfony-bundle)
• An Official AI Presence to serve: sign up for Studio
• Contributing: CONTRIBUTING.md (issues welcome, pull requests by invitation). Security reports: SECURITY.md.

Local development (wp-env)

Prerequisites: Docker Desktop and Node.js.

From the plugin directory:

npx @wordpress/env start

wp-env (configured in .wp-env.json) mounts and activates this plugin automatically on both instances. Then:

• Dev site: http://localhost:8888 (wp-admin, user admin, password password) — ships with the Plugin Check plugin.
• Isolated tests instance: http://localhost:8889 — preloads the field antagonists (Wordfence + WP Super Cache) the integration lane runs against.

Common commands:

npx @wordpress/env stop        # stop the containers
npx @wordpress/env clean all   # reset the database
npx @wordpress/env destroy     # remove the environment
npx wp-env run cli wp <cmd>    # run WP-CLI inside the container

Ref: Get started with wp-env.

Configure it (set your website URL)

In wp-admin → Settings → AI Presence: enter your Globetrotters website URL, choose a refresh cadence, and Save Changes — then Refresh now. The plugin pulls the apex artefacts from that URL and starts serving them.

Scriptable equivalent:

npx wp-env run cli wp option update gtaip_settings --format=json \
  '{"website_url":"https://your-site.example","refresh_interval":"daily"}'

Verify it's serving:

curl -I http://localhost:8888/llms.txt
curl -s http://localhost:8888/.well-known/globetrotters-apex-version.json

What the apex serves

Every file except the version marker is the published copy, byte for byte, and a required artefact of every refresh: one missing upstream fails the pull and the last good bundle keeps serving. The marker is fetched on its own and synthesized when Globetrotters serves none, so it never fails a refresh. The set is ContentTypes, a mirror of the backend's apex discovery set plus the marker.

Since 0.9.1 three aliases answer with a canonical file's bytes and headers, in place, with no redirect: /agents.md (AGENTS.md), and /.well-known/agent-skills and /.well-known/agent-skills/ (the Agent Skills index). They are the spellings AI Readiness scanners probe. A hit is reported under the path actually requested. The sitemap, the API catalog and the Link header list canonical paths only.

ApiCatalog answers /.well-known/api-catalog, an RFC 9727 API catalog: one linkset entry anchored on this site's homepage, with /llms.txt as service-doc and the MCP card, the agent card and the AI catalog as service-desc. Since 0.9.1 it also lists the MCP server as an item (remotes[0].url from the cached mcp.json, the one link on the Globetrotters host), left out when the manifest names no remote. Like the sitemap it is generated from the cache on every request, never fetched, on this site's own origin, and lists only what is cached; with none of its targets cached the path falls through to WordPress's 404. It carries the artefacts' headers (nosniff, no-store on both cache headers, Access-Control-Allow-Origin: *), is listed in /ai-sitemap.xml, and agent requests for it are reported as agent traffic.

LinkHeader adds the IANA-registered api-catalog and service-doc relations validators look for, then the HTTP twin of the head <link> tags, to the homepage, on a 200 HTML front-page response only, as a header line of its own beside the Link header core already sends:

Link: </.well-known/api-catalog>; rel="api-catalog", </llms.txt>; rel="service-doc", </.well-known/mcp.json>; rel="mcp", </.well-known/agent-card.json>; rel="agent-card", </.well-known/ai-catalog.json>; rel="ai-catalog"

It names only the targets this install serves: a cached artefact, or the API catalog once one of its targets is cached. A full-page cache serving the homepage without running PHP (WP Super Cache's static mode, for one) replays whatever headers it stored, which may not include it.

Under the subdomain_breadcrumb install profile, Breadcrumbs also prints a <link> block on wp_head. Each discovery relation points at this apex once the file is cached here (the Globetrotters copy until then), rel="alternate" is homepage-only, and rel="related" names the Globetrotters-hosted presence host itself, on every page and whatever the cache holds — once every file is served locally, it is the one relation that still points there:

<!-- Globetrotters — AI presence -->
<link rel="alternate" type="application/ld+json" href="/schema.json">
<link rel="ai-catalog" href="/.well-known/ai-catalog.json">
<link rel="mcp" href="/.well-known/mcp.json">
<link rel="agent-card" href="/.well-known/agent-card.json">
<link rel="related" href="https://ai.your-domain.example/">

Refreshing from cron, WP-CLI and the web server alike

The pulled bundle lives in a non-autoloaded option, gtaip_artefacts, so every process using the site's database reads the same copy: WP-Cron's loopback, a system cron running wp cron event run --due-now, and PHP-FPM. A persistent object cache in front of that table need not be shared, though. An APCu object-cache.php drop-in (W3 Total Cache's APCu object cache, APCu Manager…) lives inside one PHP process pool, so a pull run by WP-CLI wrote the table while PHP-FPM kept serving the bundle it had cached, indefinitely. Since 0.9.4:

• Each pull writes a stamp (gtaip_artefacts_stamp): the bundle's generation, version, time, and who pulled it — the lane (wp-cli, wp-cron or web for Refresh now) and the OS user. Behind a persistent object cache, every request that reads the bundle reads this small row past the cache, and a cached bundle of another generation is evicted and read again (a bundle cached before 0.9.4 carries no generation and is re-read until the next pull). A stamp that does not land rolls the bundle back. Without a persistent object cache, nothing extra is read.
• The web server leaves a heartbeat (gtaip_served): the version a web request actually served, written when it changes and otherwise at most hourly, never from cron or WP-CLI. The flush sends it as captureContext.servedVersion (the apex version id, e.g. "2026-10-18-050325", or null), the same key and format as the Globetrotters Symfony bundle 0.9.4. Settings → AI Presence shows the last pull's lane and user beside the served version, and flags a version served after the pull that is not the pull's.
• One refresh at a time. WP-Cron's loopback and a system cron could fire the same due refresh milliseconds apart, and each fetched the whole set. A lock row (gtaip_refresh_lock), taken with INSERT IGNORE against the options table — never add_option(), which trusts the local cache — makes a second refresher stand down at once without fetching or recording an error. A holder that died is reclaimed after ten minutes, and a holder that was only slow finds its lock gone before it stores, and discards its pull.

robots.txt

RobotsFilter appends one block to WordPress's virtual robots.txt: the AI user agents in the Globetrotters registry, a Content-Signal for them, an Agentmap: /.well-known/ai-catalog.json line while the catalog is cached (since 0.7.0), and a Sitemap: line for this host's /ai-sitemap.xml. It hooks robots_txt at PHP_INT_MAX, so it reads the file every other filter produced, and adds nothing when the site discourages search engines, is not connected, or has no bundle yet.

Naming an agent is not neutral, so by default the block changes nothing a crawler may fetch. Under RFC 9309 a crawler obeys only the group(s) naming it and falls back to User-agent: * only when none does. A named group saying Allow: / would release that agent from the wildcard rules — WordPress's own Disallow: /wp-admin/, a staging site's Disallow: / — and one added for an agent the file already names would merge with the site's rules for it and outrank them. So, reading the existing file through RobotsPolicy:

• agents the file already names are never added. Applebot counts as named when Googlebot is, since Apple documents that it follows Googlebot's group;
• every other agent inherits the wildcard group's lines, carried once in a single shared multi-User-agent group. Only a file with no wildcard rules gives each agent its own Allow: / group — which is what it already had;
• the signal is Content-Signal: search=yes, ai-input=yes, or the wildcard group's own Content-Signal when it has one;
• a user-agent section the file leaves open is closed with a pathless Disallow: — which changes nothing — before the block is appended, so the appended groups cannot join it.

Groups are read the way Google reads them: only an Allow or Disallow line ends a user-agent section, so a Content-Signal, Crawl-delay or Sitemap line between two User-agent lines leaves them sharing the rules that follow. User-agent values are matched on their product token (GPTBot/1.1 and GPTBot2 both name GPTBot), as Google's parser reads them.

tests/unit/Serving/RobotsFilterTest checks this against a port of google/robotstxt's matcher: across every fixture, decorating must not change any registry agent's allow/deny on any probed path.

Two opt-ins, both off, under Settings → AI Presence → AI crawlers in robots.txt. robots_ai_agents: allow_all gives every agent the file does not name Allow: /, overriding the wildcard rules for it (never a group the site wrote); robots_ai_train: true adds ai-train=yes. With both, and no wildcard rules, the block is byte-for-byte the backend registry's. A Content-Signal your wildcard group declares is kept either way: allow_all grants crawl permission, and does not overwrite what you have said about how your content may be used. Same behaviour and option names as the Symfony bundle's robots.ai_agents / robots.ai_train. Scripted — the option is written whole, so include every key the install already uses (see the note under Agent traffic reporting):

npx wp-env run cli wp option update gtaip_settings --format=json '{
  "website_url": "https://your-site.example",
  "robots_ai_agents": "allow_all",
  "robots_ai_train": true
}'

Agent traffic reporting

Requests for the artefact paths terminate in this WordPress install — they never reach a Globetrotters edge — so without this lane an apex install looks like it gets no agent traffic at all. Studio issues an endpoint and a token together on the apex install screen (AI Presence → Content → Apex install, the token shown exactly once); both go into wp-admin → Settings → AI Presence. Capture and flushing are both gated on the pair being present (Options::is_reporting_configured()) — an install that will never report pays nothing per served request.

Scriptable equivalent:

npx wp-env run cli wp option update gtaip_settings --format=json '{
  "website_url": "https://your-site.example",
  "refresh_interval": "daily",
  "ingest_endpoint": "https://api.globetrotters.ai/presence/analytics/server-log",
  "ingest_token": "…",
  "trusted_proxies": "",
  "trusted_proxy_cloudflare": false
}'

The endpoint must be https://: it receives the token and every captured client IP, so a cleartext one is refused on save and reads as unconfigured if written by hand.

The website URL must be https:// too: what is pulled from it is published at this site's apex. A cleartext or scheme-less one is refused on save, and if written by hand every refresh refuses it before fetching, records why, and keeps the last good bundle serving. Redirects are followed only to https, so an https source cannot hand the pull to a cleartext one: such a redirect fails the refresh.

Write the whole option rather than patching one key: wp option patch update fails with "No data exists for key" when the key isn't already present, and the defaults live in PHP, not in the stored option. The save itself is what creates the buffer table — update_option_gtaip_settings is hooked outside is_admin() precisely so a WP-CLI configuration gets the table too.

How it fits together:

• Capture — EventRecorder writes one row per served artefact request from the serve path. It swallows every Throwable: a dead buffer degrades reporting, never serving. Timestamps are always gmdate(), never current_time() (site-local time would land hits in the wrong bucket silently).
• Storage — a dedicated table, {prefix}gtaip_events (Schema), not a serialised option: concurrent artefact requests would lose events to a read-modify-write on one blob. It is created on demand (settings save, admin_init, the flush cron, activation) and every read and write is gated on Schema::is_ready() (version ≥ 1), so a DB user without CREATE degrades to "no reporting" rather than a broken site.
• Bounds — 5000 events or 512KB, whichever comes first (EventBuffer). Overflow drops the oldest and counts them; the count ships in the next envelope and clears only once that envelope is accepted, so a gap reads as a gap and not as quiet traffic. The counts live beside the table in two option rows that are only ever changed by a single SQL statement each: prunes run concurrently on the serve path, in exactly the burst that produced the drops.
• Client IP — ClientIp believes a forwarding header only when the connection itself came from a proxy listed under Trusted proxies (TrustedProxies). X-Forwarded-For is walked from the right past listed hops, and CF-Connecting-IP is read only from Cloudflare's own ranges, under the Cloudflare preset. Nothing listed means REMOTE_ADDR: a forged vendor IP is the one thing that could manufacture a verified hit.
• Flush — WP-Cron, gtaip_flush on a plugin-registered 15-minute schedule (FlushScheduler), independent of the artefact refresh cadence (which the customer may set to weekly). wp_cron() runs on init, ahead of the parse_request serve hook, so artefact traffic drives its own flushes. Past 60% of either bound, a one-off gtaip_flush_now is queued so a busy site follows traffic instead of the clock. The flush is always the cron loopback, never inline in a served response.
• Wire — POST to the configured endpoint, token as a Bearer credential, JSON body gzipped when zlib is available (IngestClient). https only, and never following a redirect: core's HTTP client re-sends every header, the token included, to whatever Location a 3xx names. Per event: UUID, UTC timestamp, canonical path, User-Agent, client IP, referer, status, byte size. No cookies, no per-visitor identifiers.

Inspecting a local install:

npx wp-env run cli wp option get gtaip_state --format=json   # flush counters, drops, last error
prefix=$(npx wp-env run cli wp db prefix | tr -d '\r' | tail -1)
npx wp-env run cli wp db query "SELECT COUNT(*) FROM ${prefix}gtaip_events"
npx wp-env run cli wp cron event list | grep gtaip

A 202 is not proof the token is good. The ingest endpoint answers 202 to a valid batch, a revoked token, an unknown install and a malformed body alike — deliberately, so it reveals nothing about which tokens exist. The status block on the settings screen (SettingsPage::render_reporting_status()) can therefore only distinguish "configured but never accepted" from "batches are being accepted"; confirm the actual numbers in Studio.

Behind a proxy or CDN

The client IP is what lets the backend forward-confirm that a claimed ClaudeBot hit really came from Anthropic. ClientIp reads CF-Connecting-IP → leftmost X-Forwarded-For → REMOTE_ADDR, but only consults the first two when the "trusted proxy" setting is on. Off by default: on a direct-to-origin site both headers are attacker-controlled, and honouring them would let anyone claim an IP inside a published vendor range and manufacture verified=true hits. The cost of the safe default is the opposite failure — behind Cloudflare with the toggle off, every hit reports Cloudflare's address and the whole install fails verification.

Page-view counts (optional, off by default)

A second, separate lane: human page views per page of the whole site, the denominator Presence Analytics otherwise cannot see. Switched on by the Count human page views on this site checkbox in the reporting section (gtaip_settings['count_page_views'], default false), and active only while reporting is configured (Options::counts_page_views()). The wire contract is the Globetrotters server-log ingest contract, § "Page-view counts (v2)".

• Script — PageViewScript prints one inline `