Webcrawler Skill

Intelligent documentation harvesting agent that recursively crawls documentation websites and extracts structured content about specific subjects.

Last Updated: 2026-01-23

Quick Start

# Crawl Python documentation about async/await
python skills/webcrawler/scripts/crawl_docs.py \
  --url "https://docs.python.org/3/library/asyncio.html" \
  --subject "asyncio" \
  --depth 2 \
  --output .tmp/docs/python-asyncio/