scraping
Scraping
Web scraping using nu-shell and browser tools for data extraction.
Prerequisites
- nu-shell installed (
nu) query webplugin installed (for HTML scraping):nu -c "plugin add query web"- Browser extension enabled (for dynamic content): Enable the
browserextension in your agent configuration
Common Tasks
Fetching Web Pages
Use http get to retrieve HTML content:
# Simple GET request
nu -c 'http get https://example.com'
More from knoopx/pi
podman
Manages containers, builds images, configures pods and networks with Podman. Use when running containers, creating Containerfiles, grouping services in pods, or managing container resources.
126jujutsu
Manages version control with Jujutsu (jj), including rebasing, conflict resolution, and Git interop. Use when tracking changes, navigating history, squashing/splitting commits, or pushing to Git remotes.
125nix-flakes
Creates reproducible builds, manages flake inputs, defines devShells, and builds packages with flake.nix. Use when initializing Nix projects, locking dependencies, or running nix build/develop commands.
54jscpd
Finds duplicate code blocks and analyzes duplication metrics across files. Use when identifying copy-pasted code, measuring technical debt, or preparing for refactoring.
45nix
Runs packages temporarily, creates isolated shell environments, and evaluates Nix expressions. Use when executing tools without installing, debugging derivations, or working with nixpkgs.
43yt-dlp
Downloads videos from YouTube and other sites using yt-dlp. Use when downloading videos, extracting metadata, or batch downloading multiple files.
42