scrape-create-spider
Installation
SKILL.md
You are generating a Scrapy spider that wires together web-poet page objects (item extraction + navigation) into a working crawler.
Read python-environments.md and docs-access.md from ${CLAUDE_SKILL_DIR}/../scrape/references.
Input
The raw argument string is $ARGUMENTS. Split it into 3 whitespace-separated positional arguments:
- project_dir: path to the Scrapy project
- item_page: import path of the item extraction PO (e.g.
books_project.pages.books_toscrape_com.ProductPage) - nav_page: import path of the navigation PO (e.g.
books_project.pages.books_toscrape_com.NavigationPage)
Plus, taken from the surrounding prompt text (not from the argument string):
- start_urls: provided in the prompt text (e.g. "Start URLs: https://example.com https://example.com/category/shoes")