WP Manifestindependent plugin directory
manifest / analytics / ghost-crawler-detector

Ghost Crawler Detector

Detects and logs AI crawler visits with a React-based analytics dashboard showing content risk scores.

by Umang Prajapati · github.com/umang48/ghost-crawler-detector · website

★ 0stars

Install

Download source zip No release zip yet

This is the repository archive. It installs, but the folder name will carry the branch suffix and updates will not flow.

Install straight from GitHub with WP-CLI:

wp plugin install https://github.com/umang48/ghost-crawler-detector/archive/refs/heads/main.zip
No built-in updater. New versions won't appear under Dashboard → Updates. Install new releases yourself, or let Git Updater track the repository.
Readme github.com/umang48/ghost-crawler-detector ↗

A WordPress plugin that detects and analyzes AI crawler visits with real-time analytics and content risk scoring.

Author: Umang Prajapati
Website: phptutorialpoints.in
WordPress Profile: umangapps48

Features

  • AI Crawler Detection: Identifies visits from GPTBot, CCBot, Claude-Web, Google-Extended, and more
  • Real-time Logging: Tracks all crawler visits with detailed information
  • React Dashboard: Modern analytics interface built with React
  • Content Risk Score: Shows which content is being indexed by AI crawlers
  • Security-First: All data sanitized, nonce verification, capability checks
  • Privacy-Focused: All data stays on your server

Installation

  1. Upload the plugin files to /wp-content/plugins/ghost-crawler-detector
  2. Activate the plugin through the 'Plugins' menu in WordPress
  3. Navigate to 'Ghost Crawler' in the admin menu

Development

Prerequisites

  • Node.js 16+
  • npm or yarn
  • WordPress 5.8+
  • PHP 7.4+

Setup

# Install dependencies
npm install

# Build for production
npm run build

# Development mode with hot reload
npm start

Build Process

The plugin uses @wordpress/scripts for building the React dashboard:

  • Source files: src/
  • Built files: build/
  • Webpack config: webpack.config.js

Security Features

  • Input sanitization using WordPress functions
  • SQL injection prevention with prepared statements
  • XSS protection with proper escaping
  • Nonce verification for all forms
  • Capability checks for admin functions
  • IP address anonymization option

WordPress Repository Guidelines

This plugin follows all WordPress.org plugin repository guidelines:

  • ✅ GPL v2 or later license
  • ✅ Proper text domain and internationalization
  • ✅ Security best practices
  • ✅ No external dependencies for core functionality
  • ✅ Proper readme.txt format
  • ✅ Uninstall cleanup script

API Endpoints

GET /wp-json/ghost-crawler/v1/stats

Returns overall statistics and crawler breakdown

GET /wp-json/ghost-crawler/v1/logs

Returns paginated crawler visit logs

GET /wp-json/ghost-crawler/v1/content-risk

Returns content risk scores for posts/pages

Database Schema

The plugin creates one custom table: wp_gcd_crawler_logs

Fields:

  • id: Primary key
  • user_agent: Full user agent string
  • crawler_type: Identified crawler name
  • ip_address: Visitor IP
  • url: Visited URL
  • post_id: WordPress post ID (if applicable)
  • referer: HTTP referer
  • detected_at: Timestamp

License

GPL v2 or later

Support

For issues and feature requests, please use the WordPress.org support forum.