Firecrawl

Overview

Firecrawl is an API that scrapes websites and returns clean, LLM-ready content. Point it at any URL and get back markdown, HTML, or structured data — no selectors to write, no anti-bot handling, no browser management. It handles JavaScript rendering, proxy rotation, and content extraction automatically. Built for feeding web content into LLMs, RAG pipelines, and data workflows.

When to Use

• Extracting website content for RAG (Retrieval-Augmented Generation)
• Converting web pages to clean markdown for LLM consumption
• Crawling entire sites and getting structured content
• Scraping without managing browsers, proxies, or anti-bot
• Extracting structured data (products, articles) with LLM-powered extraction

Instructions

Setup