Website scraping

A workflow for turning websites into structured data. Bias toward the lightest tool that works: static HTML → JSON-in-page → reverse-engineered API → browser automation, in that order. Skipping recon and reaching straight for Playwright is the single most common way to waste hours.

When to use

This skill applies to any task shaped like "given a URL or a list of URLs, produce structured records". Examples:

• Competitor price monitoring (hotels, flights, retail, rentals, student housing, SaaS)
• E-commerce product catalogue extraction
• Real-estate or job-listing harvesting
• Reviews, ratings, availability calendars
• One-off "pull this table into a spreadsheet" jobs

Out of scope: database persistence, scheduling, dashboards, downstream normalisation against a fixed schema. This skill produces JSONL — what the caller does with it is their problem.

Workflow

Follow these steps in order. Most failures come from skipping step 1.