Skill: web-crawl

Fetch a web page and turn it into clean markdown for the wiki / an LLM. Backend is a quarantined adapter (harness/web-crawl.config.yaml, verified:false) — the builtin works now, no key.

WHAT

Purpose và context

• Purpose: turn a URL (or a site) into clean markdown and land it in raw/ or a wiki draft, never raw HTML.
• Trigger (when to use):
  • User gives a URL and wants its content ("crawl this", "scrape", "fetch the page", "đọc trang này").
  • Pulling external docs/articles into raw/ or a wiki draft.
  • Bulk reading many pages for research (prefer a premium backend then).
• Non-goals: not cloning a page's UI/look (that is /web-clone); not writing straight into wiki/ (goes via /propose / ingest); not crawling auth-walled or disallowed content.

Mental model

URL → backend (WebFetch in-session · builtin urllib+regex · premium adapter firecrawl/crawl4ai/jina) → markdown → raw/ or wiki draft → /propose or ingest → wiki. The backend is a build-now-adapt-later quarantine: builtin = offline fallback, premium = the real upgrade behind one config file.