Authenticated Web Scraper

Purpose

Scrapes content from websites that require authentication (2FA, SSO, corporate login) by leveraging the user's Windows Edge browser via Chrome DevTools Protocol (CDP). Designed for WSL2 environments where Playwright/Puppeteer can't directly reach Windows browser ports.

When to Use

• Mirroring internal documentation sites behind corporate auth
• Scraping content from sites requiring 2FA/SSO that can't be automated
• Extracting structured content (text, HTML, links) from authenticated web pages
• Crawling site navigation trees and following links to a configurable depth

Architecture