LiteRT Local Chat

A privacy-first WordPress plugin that embeds a browser-side AI chat widget powered by LiteRT-LM and WebGPU.

⚠️ LiteRT-LM Web API is Early Preview. The JavaScript inference API is still in preview. This plugin wraps it defensively — see Adapter Layer below.

---

Table of Contents

• Overview
• Architecture
• Load Flow
• Configuring the Model
• LiteRT-LM Web Limitations
• Adapter Layer
• Vanilla JS Rationale
• Shortcode Reference
• Local Development
• Build & Deploy Notes
• Roadmap

---

Overview

All AI inference runs in the visitor's browser. WordPress only:

1. Stores settings (model URL, system prompt, theme, etc.).
2. Renders the secure HTML container.
3. Conditionally enqueues assets.

No PHP-side inference. No data leaves the user's device.

┌──────────────────────────────────────────────────────┐
│  WordPress Server (PHP)                              │
│  ┌──────────┐  ┌──────────┐  ┌───────────────────┐  │
│  │ Settings  │  │ Shortcode│  │ Asset Enqueue     │  │
│  │ API       │  │ Renderer │  │ (conditional)     │  │
│  └──────────┘  └──────────┘  └───────────────────┘  │
└───────────────────────┬──────────────────────────────┘
                        │ HTML + CSS + JS (static)
                        ▼
┌──────────────────────────────────────────────────────┐
│  Browser (Client)                                    │
│  ┌─────────────┐  ┌──────────────┐  ┌─────────────┐ │
│  │ frontend.js │──▶│litert-adapter│──▶│ @litert-lm  │ │
│  │ (UI + state)│  │ .js (bridge) │  │ /core (WASM)│ │
│  └─────────────┘  └──────────────┘  └──────┬──────┘ │
│                                            │ WebGPU  │
│                                            ▼         │
│                                   ┌───────────────┐  │
│                                   │ Gemma .litertlm│  │
│                                   │ (model file)  │  │
│                                   └───────────────┘  │
└──────────────────────────────────────────────────────┘

---

Architecture

Directory Structure

litert-local-chat/
├── litert-local-chat.php          # Main bootstrap, header, activation hooks
├── uninstall.php                  # Removes plugin options on uninstall
├── readme.txt                     # WordPress.org readme format
├── README.md                      # This file (technical docs)
├── CHANGELOG.md                   # Keep a Changelog format
├── composer.json                  # Dev-only: PHPCS/PHPStan config
├── package.json                   # Dev-only: lint + pot generation
├── languages/
│   └── litert-local-chat.pot      # Translation template
├── includes/
│   ├── Autoloader.php             # PSR-4 autoloader for LiteRtLocalChat\
│   ├── Plugin.php                 # Singleton container, wires all hooks
│   ├── Admin/
│   │   ├── Settings.php           # register_setting() + sanitization
│   │   └── SettingsPage.php       # Options page UI (Settings API)
│   ├── Frontend/
│   │   ├── Shortcode.php          # [litert_local_chat] registration
│   │   ├── Renderer.php           # Secure HTML container builder
│   │   └── Assets.php             # Conditional script/style enqueue
│   └── Support/
│       ├── Options.php            # Option defaults + typed getters
│       ├── Compat.php             # PHP/WP version checks
│       └── Logger.php             # Debug-gated error_log wrapper
└── assets/
    ├── css/
    │   ├── frontend.css           # Chat widget theming (light/dark/auto)
    │   └── admin.css              # Settings page styling
    ├── js/
    │   ├── frontend.js            # UI controller (vanilla JS, no build)
    │   └── litert-adapter.js      # LiteRT-LM API bridge (ES module)
    └── images/
        └── icon.svg               # Plugin icon placeholder

PHP Class Map

Class	Namespace	Responsibility
Plugin	LiteRtLocalChat	Singleton; wires Admin + Frontend hooks
Autoloader	LiteRtLocalChat	PSR-4 class loader (no Composer at runtime)
Settings	LiteRtLocalChat\Admin	register_setting() + per-field sanitization
SettingsPage	LiteRtLocalChat\Admin	Settings API form rendering
Shortcode	LiteRtLocalChat\Frontend	[litert_local_chat] handler
Renderer	LiteRtLocalChat\Frontend	Secure HTML output
Assets	LiteRtLocalChat\Frontend	Conditional enqueue logic
Options	LiteRtLocalChat\Support	Defaults + typed getters
Compat	LiteRtLocalChat\Support	Version compatibility checks
Logger	LiteRtLocalChat\Support	Debug-gated logging

---

Load Flow

sequenceDiagram
    participant WP as WordPress
    participant SC as Shortcode
    participant R as Renderer
    participant A as Assets
    participant JS as frontend.js
    participant ADP as litert-adapter.js
    participant API as @litert-lm/core

    WP->>SC: [litert_local_chat]
    SC->>R: render(overrides)
    R-->>WP: HTML container (data-litert-config)
    SC->>A: mark_rendered()
    WP->>A: wp_enqueue_scripts (priority 20)
    A->>A: was_rendered() ? → true
    A->>WP: enqueue frontend.js + frontend.css
    Note over JS: On DOMContentLoaded
    JS->>JS: Parse config from data-litert-config
    JS->>ADP: dynamic import(adapter.js)
    ADP-->>JS: adapter ready
    JS->>ADP: isSupported() (WebGPU check)
    alt Supported
        JS->>JS: Enable input, status = idle
    else Not supported
        JS->>JS: Show compat banner, disable input
    end
    Note over JS: User sends first message
    JS->>ADP: init(config, callbacks)
    ADP->>API: dynamic import(cdnUrl)
    ADP->>API: Engine.create({model, maxNumTokens})
    ADP->>API: engine.createConversation({preface})
    API-->>ADP: conversation ready
    ADP-->>JS: status = ready
    JS->>ADP: stream(prompt)
    ADP->>API: conversation.sendMessageStreaming(text)
    loop Each token chunk
        API-->>ADP: chunk.content[].text
        ADP-->>JS: onChunk(text)
        JS->>JS: Append to DOM (aria-live)
    end

---

Configuring the Model

Default Model

The plugin ships with a sensible default:

https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it-web.litertlm

Changing the Model

Go to Settings → LiteRT Local Chat → Model URL and enter any web-compatible .litertlm URL.

Available Web Models

Model	Size (approx.)	HuggingFace Repo
Gemma 4 E2B IT (web)	~2 GB	litert-community/gemma-4-E2B-it-litert-lm
Gemma 4 E4B IT (web)	~4 GB	litert-community/gemma-4-E4B-it-litert-lm

Self-Hosting the Model (Advanced)

For better privacy and caching:

1. Download the .litertlm file.
2. Upload it to your server (e.g., wp-content/uploads/models/).
3. Set the Model URL to the full URL of the hosted file.
4. Ensure CORS headers allow access if loading from a different domain.

---

LiteRT-LM Web Limitations

The LiteRT-LM JavaScript API is marked Early Preview (as of v0.13–v0.14). Key constraints:

Aspect	Status	Notes
Backend	WebGPU only	CPU backend not available for web
Model format	.litertlm (web variants only)	General .litertlm files not yet web-compatible
Modality	Text-in / text-out	No vision or audio support on web yet
Streaming	✅ Supported	sendMessageStreaming() returns a ReadableStream
Temperature	Best-effort	Not clearly documented in JS ConversationConfig
Max output tokens	Best-effort	Context length (maxNumTokens) IS supported via EngineSettings
WASM features	Relaxed SIMD + JSPI required	Asyncify fallbacks exist
Memory	High	Model files are hundreds of MB; low-RAM devices may fail

This plugin encapsulates all of these constraints in the adapter layer (see below).

---

Adapter Layer

The file assets/js/litert-adapter.js is the only place that imports @litert-lm/core. This isolation is intentional:

// frontend.js → adapter → @litert-lm/core
//     UI logic        API boundary     Preview WASM API

Benefits:

• API changes in LiteRT-LM don't break the UI — only the adapter needs updating.
• Every adapter call is wrapped in try/catch; failures return { ok: false, error } instead of throwing.
• Capability detection (isSupported()) runs before any heavy module loading.
• Temperature / maxTokens are passed defensively; if the preview API doesn't accept them, the adapter continues without error.

Adapter interface:

adapter.isSupported()          // → { supported, reasons[] }
adapter.init(config, callbacks) // → { ok, error? }
adapter.stream(prompt)          // → { ok, error? }
adapter.cancel()               // → void
adapter.dispose()              // → Promise<void>

---

Vanilla JS Rationale

This plugin uses vanilla JavaScript (no React, Vue, or build step) for v0.1.

Why:

• No build step required — Files ship as-is. No webpack, no babel, no transpilation.
• Smaller attack surface — Fewer dependencies = fewer vulnerabilities.
• Faster MVP — The chat UI is simple enough for vanilla DOM manipulation.
• Easier to audit — Anyone can read and modify the code without a toolchain.

Trade-off: If the UI grows significantly (e.g., settings panels, rich formatting, multimodal), migrating to a framework with a build step would be warranted. This is documented in the roadmap.

---

Shortcode Reference

Basic usage

[litert_local_chat]

With per-instance overrides

Attribute	Overrides	Example
title	Chat title	[litert_local_chat title="Support Bot"]
welcome	Welcome message	[litert_local_chat welcome="Ask me anything!"]
model	Model URL	[litert_local_chat model="https://example.com/model.litertlm"]

Attributes are optional and override the global settings for that instance only.

---

Local Development

Requirements

• PHP 8.1+
• WordPress 6.5+
• Node.js 18+ (for optional linting / pot generation)
• Composer 2.x (for optional PHPCS / PHPStan)

Setup

1. Clone or copy the litert-local-chat/ folder into your WordPress wp-content/plugins/ directory.
2. Activate the plugin in WordPress admin.
3. Test by adding [litert_local_chat] to a page.

Optional Tooling

# PHP linting + standards (requires Composer)
composer install
composer run lint

# JS linting (requires Node.js)
npm install
npm run lint

# Generate .pot translation file
npm run pot

Debugging

Enable Debug mode under Settings → LiteRT Local Chat. This logs:

• PHP errors to wp-content/debug.log (when WP_DEBUG is on).
• JavaScript logs to the browser console.

---

Build & Deploy Notes

No Build Step Required

The plugin ships ready-to-use. All JavaScript and CSS are static files with no compilation step.

Versioning

Versions follow Semantic Versioning. See CHANGELOG.md.

Release Packaging

# Create a distribution zip (excluding dev files)
zip -r litert-local-chat.zip litert-local-chat/ \
    -x "*/node_modules/*" \
    -x "*/vendor/*" \
    -x "*/.git/*"

Content Security Policy (CSP)

If your site uses a strict CSP, ensure the following directives allow the LiteRT-LM CDN:

script-src 'self' https://cdn.jsdelivr.net;
connect-src 'self' https://huggingface.co https://cdn.jsdelivr.net;

Or self-host @litert-lm/core and update the CDN URL setting to your own domain.

---

Roadmap

• [ ] Gutenberg block with block.json and @wordpress/scripts build
• [ ] Persistent chat history (opt-in, via localStorage)
• [ ] React UI (if complexity warrants a framework migration)
• [ ] Self-hosted model upload via WordPress Media Library
• [ ] CSP-friendly mode (bundle LiteRT-LM core locally instead of CDN)
• [ ] Markdown rendering in assistant responses
• [ ] Copy / regenerate message actions
• [ ] Multi-language UI (full .pot coverage and translations)
• [ ] Model warm-up on page load (optional, configurable)
• [ ] Device memory pre-check threshold setting

---

License

GPL-2.0-or-later. See license.

The LiteRT-LM library is licensed under Apache-2.0 by Google LLC.