Web Crawler

Playwright-based headless browser for crawling JavaScript-heavy sites, taking screenshots, clicking buttons, and running UX audits. Works on SPAs, auth-walled pages, and anything that needs a real browser.

Quick Reference

Single page screenshot

python3 skills/web-crawler/scripts/crawl.py https://example.com

Full UX audit (crawls all internal pages)

python3 skills/web-crawler/scripts/crawl.py https://example.com --audit --max-pages 30