Direct Crawler

A WP-CLI command to get posts from the HTML site

★ 1stars

Install

Download source zip No release zip yet

This is the repository archive. It installs, but the folder name will carry the branch suffix and updates will not flow.

Install straight from GitHub with WP-CLI:

wp plugin install https://github.com/tmeister/direct-crawler/archive/refs/heads/main.zip

No built-in updater. New versions won't appear under Dashboard → Updates. Install new releases yourself, or let Git Updater track the repository.

The plugin is a simple crawler implementation to get content from an external website and import them as a WordPress post.

The plugin doesn't have a UI. It needs to be run n a command line like a WP CLI Command.

Run composer install to install the PHP dependecies
Add the post URLs from where you want to get the content to the posts.csv file, one URL per line.
On the Parser.php file, update the $siteUrl variable using the final site domain.
Update the CSS selectors on the Parser.php file to the site's CSS selector to get the correct values. Ex. Where to get the H1 info, the date, etc.
Run the WP CLI Command via the command line.

wp direct get