Web Scraper

Four scripts, zero API keys. All output is JSON by default.

Dependencies: requests, beautifulsoup4, playwright (with Chromium). Optional: pdfplumber or PyPDF2 for PDF text extraction.

Install: pip install requests beautifulsoup4 playwright && playwright install chromium

1. Search the Web

python3 scripts/google_search.py "query" --pages N --engine ENGINE

• --engine — duckduckgo (default), brave, or google
• Returns [{title, url, snippet}, ...]

2. Read a Page (one-shot)