Firecrawl Monitoring

Firecrawl is an external fetch + change-detection backend for scrape sources our own pipeline can't reach: Firecrawl scrapes the page on a schedule (its proxies clear the anti-bot challenge), diffs each check, runs an AI meaningfulness judge, and POSTs us a webhook that becomes a release. Full architecture: docs/architecture/firecrawl-monitoring.md.

It is not a new source type — it's a per-source toggle on an existing scrape source, stored under source.metadata.firecrawl. source.url stays the human-readable page.

Prod-only. The Firecrawl bindings exist only in production (staging shares prod's Secrets Store, so a staging sync would mutate prod monitors — never add the bindings to [env.staging]).

When to use it

• The page is behind a Cloudflare Managed Challenge (or similar) that our Browser Rendering can't clear — render "succeeds" with a challenge shell, no_change / 0 releases on a page that's clearly updating.
• A client-rendered page keeps failing the render path and there's no feed, no .md view, no GitHub source (finding-changelogs exhausted its ladder).

When NOT to use it:

• A feed/GitHub/plain-scrape source that works — Firecrawl costs credits per check; the in-repo pipeline is free.
• First resort on a rendering hiccup. Run the render dry-run first (releases source fetch <source> --dry-run) — a populated candidate count means the normal pipeline can see the page and something else is wrong.
• Staging, or any environment without the prod bindings.