Digikala Product Scraper

🔍 A WordPress plugin that extracts product information from Digikala product listing pages and imports them into WooCommerce.

✨ Features

• Smart Scraping: Automatically detects and extracts product information from Digikala listing pages
• WooCommerce Integration: Seamlessly imports products into your WooCommerce store
• Multi-page Support: Can scrape multiple pages from a single listing
• Image Import: Downloads and imports product images
• Category Management: Auto-creates WooCommerce categories or assigns to existing ones
• Progress Tracking: Real-time progress monitoring during import
• Import History: Complete logging and history of all import operations
• Configurable Settings: Adjustable delays, page limits, and import options
• Responsive UI: Modern, mobile-friendly admin interface

📚 Table of Contents

• Requirements
• Installation
• Usage
• Configuration
• Admin Interface
• Technical Details
• Troubleshooting
• Security Considerations
• Legal and Ethical Use
• Support
• Changelog
• License

✅ Requirements

• WordPress 5.0 or higher
• PHP 7.4 or higher
• WooCommerce plugin (active)
• cURL support
• DOMDocument support

🚀 Installation

1. Upload the digikala-product-scraper folder to your /wp-content/plugins/ directory
2. Activate the plugin through the Plugins menu in WordPress
3. Navigate to Digikala Scraper in your admin menu

🛠️ Usage

Basic Import Process

1. Find a Digikala listing page: Go to Digikala and navigate to a category page, search results page, or any page that shows multiple products
2. Copy the URL: Copy the complete URL from your browser's address bar
3. Start Import: Go to WordPress Admin -> Digikala Scraper -> Scrape Products
4. Paste the URL: Enter the Digikala URL in the form
5. Configure Settings: Adjust the number of pages and delay as needed
6. Select Category: Choose a WooCommerce category or leave as default to auto-detect
7. Start Import: Click Start Import to begin the scraping process

What Gets Imported

• Product Title: Main product name
• Price: Current selling price
• Regular Price: Original price (if on sale)
• Product Images: High-quality product images
• Brand: Product brand information
• Rating: User rating (if available)
• Review Count: Number of reviews
• Product URL: Link to original Digikala page
• SKU: Product identifier

Import Options

• Auto-create categories: Automatically creates WooCommerce categories based on Digikala URLs
• Import images: Downloads and imports product images from Digikala
• Import descriptions: Generates product descriptions from scraped data
• Import attributes: Creates product attributes for brand, rating, etc.

⚙️ Configuration

Scraping Settings

• Scraping Delay: Time between requests (1-10 seconds)
• Max Pages: Maximum pages to scrape per listing (1-20)
• User Agent: Custom user agent string for HTTP requests

Import Settings

• Category Assignment: Choose between auto-detection and manual selection
• Image Import: Enable/disable image downloading
• Description Generation: Control automatic description creation
• Attribute Import: Manage product attribute creation

🧩 Admin Interface

Dashboard

• Overview of import statistics
• Quick access to main functions
• Recent import activity
• System information

Scrape Products

• Main import interface
• URL input and validation
• Category selection
• Import configuration
• Real-time progress tracking
• Results display

Settings

• Plugin configuration
• Import options
• System requirements check
• Help documentation

Import History

• Complete log of all imports
• Success/failure statistics
• Detailed import results
• Export functionality
• Bulk operations

🧠 Technical Details

Database Tables

The plugin creates a custom table wp_dps_scraping_logs to store:

• Import URLs and status
• Products found and imported
• Error messages and timestamps
• Performance metrics

File Structure

digikala-product-scraper/
├── digikala-product-scraper.php          # Main plugin file
├── includes/
│   ├── class-digikala-scraper.php        # Core scraping logic
│   └── class-woocommerce-importer.php    # WooCommerce integration
├── admin/
│   ├── dashboard.php                     # Main dashboard
│   ├── scrape.php                        # Import interface
│   ├── settings.php                      # Configuration
│   └── history.php                       # Import logs
└── README.md                             # This file

AJAX Endpoints

• dps_scrape_digikala: Main scraping function
• dps_get_scraping_progress: Progress monitoring
• dps_test_connection: URL validation
• dps_cleanup_logs: Log cleanup
• dps_get_log_details: Detailed log information

🧯 Troubleshooting

Common Issues

1. Import Fails: Check if the URL is accessible and contains product listings
2. No Products Found: Verify the page structure hasn't changed
3. Images Not Importing: Ensure media uploads are enabled and writable
4. Slow Performance: Increase scraping delay to be more respectful to servers

Error Messages

• "Connection failed": Check internet connection and URL validity
• "Not a Digikala page": Verify the URL is from digikala.com
• "WooCommerce not active": Activate WooCommerce plugin first
• "Permission denied": Ensure you have administrator privileges

Performance Tips

• Start with small page limits (1-3 pages)
• Use reasonable delays (3-5 seconds)
• Monitor server resources during large imports
• Clean up old logs regularly

🔒 Security Considerations

• The plugin only scrapes publicly accessible pages
• Respects robots.txt and server guidelines
• Configurable delays prevent server overload
• User authentication required for all operations
• Nonce verification for all AJAX requests

⚖️ Legal and Ethical Use

• Respect Terms of Service: Always review and comply with Digikala's terms
• Rate Limiting: Use appropriate delays between requests
• Data Usage: Only scrape publicly available information
• Server Respect: Do not overwhelm servers with excessive requests

🙋 Support

For issues and questions:

1. Check the troubleshooting section above
2. Review WordPress error logs
3. Verify plugin requirements are met
4. Test with a simple URL first

📝 Changelog

Version 1.0.0

• Initial release
• Basic Digikala scraping functionality
• WooCommerce integration
• Admin interface
• Import logging and history

📄 License

This plugin is licensed under the GPL v2 or later.

🙌 Credits

Developed for WordPress and WooCommerce integration with Digikala product data extraction capabilities.

---

Note This plugin is designed for educational and legitimate business purposes. Always ensure compliance with applicable laws and website terms of service.