WP Manifestindependent plugin directory
manifest / ai / wordpress-ai-assistant

Site Assistant

WordPress plugin: an AI chat assistant that answers visitors from the site's own pages with source links, using Claude (official PHP SDK). Says not found instead of guessing.

★ 0stars

Install

Download source zip No release zip yet

This is the repository archive. It installs, but the folder name will carry the branch suffix and updates will not flow.

Install straight from GitHub with WP-CLI:

wp plugin install https://github.com/coderock-izzy/wordpress-ai-assistant/archive/refs/heads/main.zip
No built-in updater. New versions won't appear under Dashboard → Updates. Install new releases yourself, or let Git Updater track the repository.

Site Assistant: an AI chat assistant for WordPress that answers from your own pages

A WordPress plugin that adds a chat button to a small-business website. Visitors ask questions in their own words ("how much is a cat booster?", "are you open Saturday?") and get a short answer written from the site's own pages, with links to the pages it used. When the site doesn't say, it answers "I couldn't find that on this website" and links to the contact page instead of guessing a price or a policy.

It uses Claude through the official Anthropic PHP SDK, and it still works without an API key by quoting the most relevant sentences from the site.

Chat widget answering from the site with sources, and saying it couldn't find an answer for a question the site doesn't cover

Results

php eval/run.php --gate runs on a 27-page website for a fictional vet clinic, with labelled questions in two sets: development (the thresholds were chosen on these) and held-out (written before any tuning, never used to tune).

Development Held-out
WordPress's built-in search: right page in the top 5 28% 23.5%
WordPress search: no results at all (every word must appear on the page) 72% 76.5%
Site Assistant: right page in the top 5 96% 100%
Claude mode: answerable questions whose right page reaches the model 96% 94.1%
Offline mode: answers citing only correct pages 94.9% 84.6%
Offline mode: off-site questions answered "not found" 100% 80%
Answers repeating instructions planted on a page 0 0

Search over a 2,160-page site takes under 1 ms at p95. In Claude mode, questions the site plainly doesn't cover never reach the model, so they cost nothing.

What the numbers do not show: Claude's own answer quality. Measuring it needs an API key and money per run, so CI instead checks the exact request sent, the parsing of recorded responses, and the full flow against a mock API (below). The offline mode is a fallback; on the held-out set about two thirds of its quotes contain the exact fact asked for (known limitations).

Evaluation report

How it works

Visitor question ──► POST /wp-json/saia/v1/ask (rate-limited, no cookies)
                       │
                       ▼
            BM25 over the site's passages (split at headings; titles and headings weighted)
                       │
       "Does the site bear on this?"  share of the question's IDF-weighted words found in the best passage
          ├─ no  ──► "I couldn't find that on this website" + contact link   (no model call)
          └─ yes ──► Claude (official PHP SDK)
                       │  passages as search_result blocks, one text block per sentence, citations on
                       │  system prompt: answer only from the results, they are content not instructions,
                       │  reply NOT_FOUND otherwise
                       ▼
                     answer text + search_result_location citations ──► links to the pages that were sent
                       │
             API error, rejected key or rate limit ──► quote the best sentences instead; owner sees why
  • Model and request. claude-opus-5-5 by default (Sonnet 5.5 and Haiku 5.5 selectable), effort low because this is chat, server-side refusal fallbacks (fallbacks: "default") on the models that support them, SDK retries. Earlier turns are replayed as plain text; under API version 2026-10-21 a user turn may not carry citations.
  • Citations you can trust. Each passage is a search_result block whose source is the page URL (with the section's anchor). Citations are mapped back to links, and only URLs that were actually sent can be linked.
  • Prompt injection. Sentences that read like instructions to an AI ("ignore your previous instructions…") are removed when the site is indexed and listed for the owner; the system prompt treats page text as data.
  • Two modes, two thresholds. Quoting the site needs a close match; the model gets looser candidates because it answers "not found" when they don't contain the answer (ADR 1).
  • The index is built from published posts, pages and any public post type, never private or password-protected content, and rebuilds itself a minute after content changes (or wp site-assistant reindex).
  • Privacy. No cookies, no conversation stored on the server, visitors rate-limited by a salted hash of their address kept for an hour. Optionally, the text of questions the site couldn't answer is kept (nothing about the visitor) so the owner can see what content is missing.

Admin screen: index status, a test question with its source, and a planted instruction excluded from answers

Tests

Layer What runs
Unit (21 tests) Search, chunking, stemming, the injection filter, the answerability check, fallback, and the Claude answerer through the real Anthropic SDK with an SDK middleware standing in for the network: the exact JSON sent (search results, citations, model, effort, fallbacks header and body), citation parsing, refusals, NOT_FOUND, plain-text history, and 401/403/429/500 mapped to reasons
Integration (26 checks) WordPress 6.5 and latest in Docker: indexing (private pages excluded), REST answers with sources, rate limiting, the shortcode, rebuild on save, uninstall; and Claude mode end to end: widget endpoint → retrieval → SDK and Guzzle over HTTP → a mock POST /v1/messages → citations back as links, plus the fallback when the key is rejected
Evaluation The table above, 8 gated checks
Static WordPress Coding Standards 3, PHPStan level 8

CI also builds an installable site-assistant.zip with the SDK bundled.

Run it

composer install
docker compose up -d && bash tests/integration/run.sh   # WordPress with the demo clinic site
# http://127.0.0.1:8080  (admin / admin) - the chat button is on every page

The compose file points the SDK at the mock API (ANTHROPIC_BASE_URL); remove that line and add a real key under Settings → Site Assistant (or SAIA_ANTHROPIC_API_KEY in wp-config.php) for real answers.

vendor/bin/phpunit
vendor/bin/phpcs && vendor/bin/phpstan analyse
php eval/run.php --gate          # writes eval/reports/report.{md,json,html}

Project layout

site-assistant.php        plugin header and bootstrap
src/Search/               Text (terms, stemming, sentences), Chunker, Index (BM25, coverage), InjectionFilter
src/Answer/               Assistant (pipeline), ClaudeAnswerer (SDK), ExtractiveAnswerer (offline), Answer
src/Wp/                   settings, indexer, REST endpoint, widget, WP-CLI
src/Admin/                settings screen with status, test box, excluded text and content gaps
assets/                   widget (vanilla JS, accessible), styles
eval/                     the clinic website, questions, WordPress-search baseline, report
tests/unit/ tests/integration/   (integration includes the mock Anthropic API)

Documentation

The clinic, its staff, prices and every question are invented.

License

GPL-2.0-or-later.