AgentReady – llms.txt & Markdown for AI Agents

A WordPress plugin that serves an llms.txt index and a clean Markdown version of every page, so AI agents can read your content.

The Markdown is built from the rendered page, so ACF fields, Gutenberg blocks, Elementor layouts and classic editor content are all included, whatever the theme.

Features

• /llms.txt, a Markdown index of your pages.
• A .md version of every page, for example /about.md.
• All public post types enabled by default, with a setting to exclude any of them.
• Hide individual pages from the editor, or exclude by ID, URL pattern or taxonomy term.
• Per-page description override for llms.txt.
• Preview tool to test the generated Markdown for any URL.
• Works behind HTTP Basic authentication.

URLs

/about/index.html.md, /about/index.md and /index.md are accepted as aliases. Markdown responses send Content-Type: text/markdown, a canonical Link header to the HTML page, X-Robots-Tag: noindex, and support ETag and If-Modified-Since. Each page also gets <link rel="alternate" type="text/markdown"> in its head.

Custom Permalinks

Works with default WordPress permalinks and with the Custom Permalinks plugin. A page with a custom URL gets its Markdown at that URL plus .md (/company/about-us.md), llms.txt and the discovery tag use it, and .md requests for the original WordPress URL (/about.md) are redirected to it. URL patterns in the exclusion list match the custom path.

Requirements

• PHP 8.0+ with the DOM extension, WordPress 6.0+.
• Pretty permalinks. Plain permalinks (?p=123) are not supported. Nothing is served for them and no Markdown is generated for URLs that contain a query string.

How it works

1. A request for /about.md is caught early (on parse_request), so no rewrite rules are added and nothing needs flushing.
2. The post is resolved from the path and checked for eligibility: published, not password protected, post type enabled, not excluded.
3. The page is fetched from its own public URL as a logged-out visitor, so theme templates, ACF output and page builders are included. If that fails, the post content is used instead.
4. The main content is selected (main, [role="main"], #content, #main, .site-content, article, then body). Navigation, sidebars, forms, scripts, hidden elements and site-level headers and footers are removed. Headers and footers inside an article, main or section are kept. The result is converted to Markdown with a metadata header.

Settings

Settings > AgentReady has post types, exclusions, the content selector, extra elements to exclude (CSS selectors, added to the built-in list), HTTP Basic authentication, and a Preview tab. The Preview tab can try a different content selector or extra excludes without saving them.

HTTP Basic credentials are stored encrypted with a key derived from the site's auth salt. To keep them out of the database, define them in wp-config.php:

define( 'AGENTREADY_HTTP_AUTH_USER', 'user' );
define( 'AGENTREADY_HTTP_AUTH_PASSWORD', 'password' );

Hooks

Excluding from code

// Leave out one page.
add_filter( 'agentready_is_eligible', function ( $eligible, $post ) {
    return 'private-docs' === $post->post_name ? false : $eligible;
}, 10, 2 );

// Leave out a whole post type.
add_filter( 'agentready_excluded_post_types', function ( $types ) {
    return array_merge( $types, array( 'product' ) );
} );

// Hide the editor box for everyone.
add_filter( 'agentready_show_meta_box', '__return_false' );

Installation

Composer:

composer require wpackagist-plugin/agentready

Or copy the plugin folder to wp-content/plugins/agentready and activate it.

Development

composer install

Every pull request runs these checks plus Plugin Check.

License

GPL-2.0-or-later. See LICENSE.