Olostep - Web Scraping & Crawling API

Powerful web scraping, crawling, and AI-powered content extraction.

Capabilities

• Create Scrape: Initiate a web page scrape
• Create Answer: The AI will perform actions like searching and browsing web pages to find the answer to the provided task
• Maps: This endpoint allows users to get all the urls on a certain website
• Start Crawl: Starts a new crawl
• Start Batch: Starts a new batch
• Batch Items: Retrieves the list of items processed for a batch
• Crawl Info: Fetches information about a specific crawl
• Crawl Pages: Fetches the list of pages for a specific crawl
• Get Answer: This endpoint retrieves a previously completed answer by its ID
• Get Scrape: Can be used to retrieve response for a scrape
• Batch Info: Retrieves the status and progress information about a batch
• Retrieve Content: Retrieve page content of processed batches and crawls urls