Rust Web Crawler (rcrawler)

High-performance web crawler built in pure Rust with production-grade features for fast, reliable site crawling.

When to Use This Skill

Use this skill when the user requests:

• Web crawling or site mapping
• Sitemap discovery and analysis
• Link extraction and validation
• Site structure visualization
• robots.txt compliance checking
• Performance-critical web scraping
• Generating interactive web reports with graph visualization

Core Capabilities