web-fetch
Fetch web content as clean markdown using markdown-native endpoints, selector-based HTML extraction, or bundled fallback parsing.
- Prioritizes markdown-native responses (content-type: text/markdown) before falling back to HTML extraction
- Includes pre-configured selectors for common documentation sites (Anthropic, MDN, GitHub) and a generic fallback for article/main content regions
- Provides html2markdown with CSS selector support for fine-grained content isolation, excluding navigation, headers, footers, and scripts
- Bundles a Bun-based fallback parser for sites where selector-based extraction fails or produces poor output
- Requires curl, html2markdown, and bun; includes troubleshooting guidance for markdown detection, selector testing, and client-rendered content limitations
Web Content Fetching
Fetch web content in this order:
- Prefer markdown-native endpoints (
content-type: text/markdown) - Use selector-based HTML extraction for known sites
- Use the bundled Bun fallback script when selectors fail
Prerequisites
Verify required tools before extracting:
command -v curl >/dev/null || echo "curl is required"
command -v html2markdown >/dev/null || echo "html2markdown is required for HTML extraction"
command -v bun >/dev/null || echo "bun is required for fetch.ts fallback"
Install Bun dependencies for the bundled script:
More from 0xbigboss/claude-code
react-best-practices
Use when reading or writing React components (.tsx, .jsx files with React imports).
2.6Ktypescript-best-practices
Use when reading or writing TypeScript or JavaScript files (.ts, .tsx, .js, tsconfig.json).
2.1Kpython-best-practices
Use when reading or writing Python files (.py, pyproject.toml, requirements.txt).
1.3Kzig-best-practices
Use when reading or writing Zig files (.zig, build.zig, build.zig.zon).
349nix-best-practices
Use when working with Nix flakes, overlays, shell.nix, or flake.nix files.
318openai-image-gen
Use when generating images, graphics, icons, or visual assets via OpenAI DALL-E 3 API. Requires OPENAI_API_KEY.
288