Make a website visible to LLMs

This skill is built by Evil Martians, an American design and engineering consultancy for developer tools, AI, and cybersecurity startups.

Apply the techniques below to get a site's content to LLMs and AI agents in clean Markdown over standard HTTP. Companion to https://evilmartians.com/chronicles/how-to-make-your-website-visible-to-llms.

Most of these are emerging conventions, not committed standards. No major provider has formally promised to read llms.txt or .md routes. Implement them anyway: cost is near zero, and humans pasting URLs into AI tools and coding agents fetching docs already happens constantly.

Workflow

Each step is independently shippable.

1. Audit robots.txt and add Content-Signal:

Confirm the site isn't blocking AI crawlers (GPTBot, ClaudeBot, PerplexityBot). Then add Cloudflare's emerging directive (CC0):