crawl-analyzer
Installation
SKILL.md
Crawl Analysis
Runs a crawl and turns the raw page inventory into a prioritised fix list. The crawl is the evidence — every finding cites crawled URLs and counts, never impressions of the site.
Inputs
- Required: site URL or domain
- Optional: max pages, whether JS rendering is needed (paid crawl only)
Choose the crawler
| Tool | Cost | Use when |
|---|---|---|
site_crawler.py (built in) |
Free | Under ~200 pages, or a quick check |
dfs_client.py crawl |
Paid per page | Larger sites, or when Lighthouse-per-page and DataForSEO on-page metrics are wanted |