Paywall & Subscription Content Access Audit

Audit subscription-gated publications, research portals, newsletters, and premium content platforms to ensure search engines and AI answer engines can discover, understand, and cite authoritative articles without triggering cloaking penalties or unauthorized full-text scraping.

Workflow

1. Audit Schema.org Paywall Markup Conformance:

   • Inspect JSON-LD on sample paywalled pages for NewsArticle, Article, or CreativeWork entities [GOOGLE-PAYWALL-SCHEMA-01].
   • Verify isAccessibleForFree: "False" (or boolean false) [SCHEMA-ISACCESSIBLEFORFREE-01].
   • Verify hasPart contains WebPageElement with isAccessibleForFree: "False" and valid cssSelector matching the gated content wrapper [GOOGLE-PAYWALL-SCHEMA-01].

2. Audit Lead-In / Preview Snippet Server-Side Renderability:

   • Verify that the public lead paragraph, headline, author, and date exist in the raw HTML payload (curl -s "$URL").
   • Ensure the public lead-in is not hidden with client-side JavaScript or CSS that causes AI crawlers to perceive the page as empty or cloaked.

3. Separate AI Training vs. AI Search/Citation Crawler Policies:

   • Audit robots.txt for separation of AI training bots (GPTBot, Google-Extended, Applebot-Extended) versus AI search citation bots (OAI-SearchBot, PerplexityBot, Claude-SearchBot).
   • Flag configurations that accidentally block citation bots, preventing paywalled research from being cited in AI search results.