crawl4ai
Installation
SKILL.md
Crawl4AI Skill
Complete guide for Crawl4AI - the #1 trending GitHub repository for AI-powered web crawling and scraping. Generate clean, AI-ready markdown from any website with intelligent extraction strategies.
Quick Reference
Core Capabilities
| Feature | Description |
|---|---|
| Clean Markdown | AI-optimized markdown output from any webpage |
| LLM Extraction | Use GPT-4, Claude, or local models for structured extraction |
| CSS/XPath Extraction | Fast, token-efficient data extraction with selectors |
| Browser Automation | Full Chromium control with JavaScript execution |
| Adaptive Crawling | AI-guided crawling that follows relevant links |
| Session Management | Persistent browser state across requests |
| Parallel Crawling | Efficient multi-URL processing |
| RAG Integration | Built for retrieval-augmented generation pipelines |