Firecrawl Observability

Overview

Monitor Firecrawl web scraping and crawling jobs for success rates, credit consumption, and extraction quality. Key signals include crawl job completion rate, pages scraped per credit, scrape latency (single page vs full crawl), credit burn velocity, and extraction success rate (did the markdown/structured data extraction return useful content).

Prerequisites

• Firecrawl account with API access
• Webhook endpoint for job status callbacks
• Metrics backend for tracking

Instructions

Step 1: Track Crawl Job Status via Webhooks

// firecrawl-webhook-handler.ts
app.post('/webhooks/firecrawl', (req, res) => {
  const { jobId, status, pagesScraped, creditsUsed, failedUrls } = req.body;
  emitCounter('firecrawl_jobs_total', 1, { status });
  emitGauge('firecrawl_pages_per_job', pagesScraped, { job: jobId });
  emitCounter('firecrawl_credits_used', creditsUsed);