headless-web-scraping
Installation
SKILL.md
Headless Web Scraping
Extract data from web pages that resist simple HTTP requests — JS-rendered content, Cloudflare-protected sites, and dynamic SPAs — using scrapling's three-tier fetcher architecture and CSS-based data extraction.
When to Use
- Target page requires JavaScript rendering (SPA, React, Vue)
- Site has anti-bot protections (Cloudflare Turnstile, TLS fingerprinting)
- You need structured extraction of multiple elements via CSS selectors
- Simple
WebFetchorrequests.get()returns empty or blocked responses - Extracting tabular data, link lists, or repeated DOM structures at scale
Inputs
- Required: Target URL or list of URLs to scrape
- Required: Data to extract (CSS selectors, field names, or description of target elements)
- Optional: Fetcher tier override (default: auto-select based on site behavior)
Related skills