WP Screaming Frog Spider Cleaner

Batch SEO hygiene tasks for WordPress sites using curated Screaming Frog CSVs via WP-CLI, with an intuitive admin interface to generate the commands and review the logs.

Description

What this plugin does

WP Screaming Frog Spider Cleaner runs large-scale SEO hygiene tasks on WordPress content based on CSV files exported from Screaming Frog SEO Spider. The heavy processing runs exclusively via WP-CLI (no HTTP requests), so it can safely process sites of any size without timeouts.

It ships with four tasks:

• Status Code — replaces 30x redirected URLs with their final destination in the post content, and removes images / de-references links pointing to 404/5xx URLs.
• Multiple H1 — converts extra H1 headings in the content to H2, keeping a single H1 per page.
• Images without ALT — fills missing alt attributes on images using the file name as a readable fallback.
• Images without width/height — fills missing width/height attributes on images (from the file name, the attachment metadata, the local file, or a remote fallback) to reduce layout shift (CLS).

Why use this plugin

• Processes millions of rows safely through streaming CSV reads and batching, with automatic checkpoints and locking.
• Keeps the site responsive: nothing runs on the front end or during normal admin usage — only when you invoke the WP-CLI command.
• Provides a dry-run mode so you can review every change in a log before writing anything to the database.
• Writes directly to the database in a controlled way, bypassing expensive hooks for maximum throughput.

Admin interface

The admin screen (under Tools → WP Screaming Frog Spider Cleaner) lets you:

• Upload the exported CSV files (or drop them via FTP) into a monitored folder.
• Select the task and the relevant CSVs.
• Generate the ready-to-copy WP-CLI command (pre-filled with --dry-run).
• Review the tail of the most recent log for each task.

Requirements

• WP-CLI available on the server.
• Access to run WP-CLI commands (SSH or equivalent).
• CSV files exported from Screaming Frog SEO Spider.

How it works

1. You export the relevant reports from Screaming Frog SEO Spider (e.g. redirects, inlinks, images).
2. You upload those CSVs on the admin screen (or via FTP) to the monitored folder: wp-content/uploads/wpsfc/files/.
3. You pick the task and the CSVs, then copy the generated WP-CLI command.
4. You run the command with --dry-run first and review the log in wp-content/uploads/wpsfc/logs/.
5. Once satisfied, you remove --dry-run and run it for real.

The runner streams the CSV in batches, resolves the affected posts, applies the changes to the content, and saves periodic checkpoints so the job can be resumed if interrupted.

Installation

1. Upload and activate the plugin like any standard WordPress plugin.
2. Make sure WP-CLI is installed and available on the server.
3. Go to Tools → WP Screaming Frog Spider Cleaner.
4. Upload the CSV files exported from Screaming Frog (or place them in wp-content/uploads/wpsfc/files/).
5. Select the task and CSVs, copy the generated command, and run it via WP-CLI with --dry-run.
6. Review the log, then re-run without --dry-run to apply the changes.

WP-CLI commands

# Status Code (redirects + broken links/images)
wp wpsfc status-code --status-csv=<path> --inlinks-csv=<path> --batch=2000 --dry-run

# Multiple H1 -> H2
wp wpsfc multi-h1 --csv=<path> --batch=2000 --dry-run

# Images without ALT
wp wpsfc missing-alt --csv=<path> --batch=3000 --dry-run

# Images without width/height
wp wpsfc missing-sizes --csv=<path> --batch=3000 --dry-run

Common flags

• --dry-run: simulates execution without writing to the database.
• --resume: resumes from the last saved checkpoint.
• --force: ignores an existing lock and restarts the task.
• --batch=<n>: number of CSV rows per batch (higher = more throughput and more memory).
• --verbose: detailed log per updated item (more log IO).

Checkpoint and lock refresh are handled automatically (no command parameters required).

Frequently Asked Questions

Do I need WP-CLI?

Yes. All the heavy processing runs through WP-CLI to guarantee performance and avoid HTTP timeouts. The admin screen only generates the command and shows logs.

Will it change my content without confirmation?

No. Every command is generated with --dry-run by default. In dry-run mode nothing is written to the database — you only get a log of what would change. You must explicitly remove --dry-run to apply changes.

Does the "missing width/height" task work with external images?

Yes. It first tries to read the size from the file name, then from the attachment metadata, then from the local file, and finally falls back to a safe remote request (using wp_safe_remote_get, which protects against SSRF) to read the image header.

Where are the CSV files stored?

In wp-content/uploads/wpsfc/files/. You can upload them from the admin screen or drop them via FTP.

Where are the logs?

In wp-content/uploads/wpsfc/logs/, one file per task/run. The most recent log for each task is also shown on the admin screen.

Can I interrupt a job and resume it later?

Yes. The runner saves periodic checkpoints. Run the same command with --resume to continue from where it stopped.

Compatibility / Requirements

• Recommended minimum: WordPress ≥ 5.5, PHP ≥ 7.4
• Tested with: WordPress 6.8
• WP-CLI: required
• CSV files must be exported from Screaming Frog SEO Spider with the expected columns per task.

Screenshots

1. Admin screen under Tools → WP Screaming Frog Spider Cleaner: upload CSVs, pick a task, generate the WP-CLI command, and review logs.

Changelog

1.0.0

• Initial public release.
• Tasks: Status Code, Multiple H1, Images without ALT, Images without width/height.
• Streaming CSV processing with batching, checkpoints, and locking.
• Dry-run mode, resume, force, verbose, and configurable batch size.
• Admin interface to upload CSVs, generate WP-CLI commands, and view logs.

Upgrade Notice

1.0.0

First stable release. Requires WP-CLI to run the processing commands.

Privacy

This plugin does not collect or transmit any personal data. The "missing width/height" task may perform outbound HTTP requests to read image headers when an image size cannot be determined locally; those requests only fetch the referenced image and send no personal data.