Site Content Catalog

Build a content inventory for any domain — keyless. The script discovers pages via robots.txt Sitemap directives, sitemap.xml (with sitemap-index recursion), common sitemap locations, and RSS/Atom feeds; classifies each URL by content type; clusters by URL slug; computes publishing cadence; and optionally deep-reads the top N pages for stats.

When to use

• "Catalog all the content on [domain]."
• Before an SEO audit, content-gap analysis, or as content-brief-factory's existing- coverage check.
• When you need funnel-stage mapping or publishing-pattern stats for a competitor/own site.

How to run

Bundled script scripts/site_catalog.py (Python 3 stdlib only — no install):