llms.txt explained: what it is and who reads it

llms.txt is a markdown file you publish at the root of your site — https://example.com/llms.txt — that gives AI assistants a short, curated guide to what your site is and which pages matter most. Think of it as a counterpart to robots.txt and sitemap.xml, but written for a language model instead of a search crawler: instead of listing every URL or setting crawl rules, it hands an assistant a hand-picked summary and a set of links in the order you'd want them read.

The idea comes from the llmstxt.org proposal. The pitch is simple: models work from limited context windows, so a concise, structured pointer to your best content is easier to consume than your full navigation, your JavaScript, and your marketing chrome. You decide what's canonical; the model doesn't have to guess.

What does an llms.txt file look like?

The format is ordinary markdown with a light convention. The first non-blank line is an H1 with your site or product name. A blockquote line gives a one-sentence summary. Then ## sections group links to your key pages, each as a normal markdown link:

# Acme Audio

> Acme Audio makes wireless headphones and speakers focused on
> noise cancellation and long battery life.

## Products
- [Wireless Noise-Cancelling Headphones](https://www.acme.com/audio/headphones/wireless-nc)
- [Portable Speakers](https://www.acme.com/audio/speakers)

## Company
- [About Acme](https://www.acme.com/about)
- [Support](https://www.acme.com/support)

There's an optional companion file, llms-full.txt, that inlines the full text of those pages rather than just linking to them — useful when you'd rather an assistant read the content directly than fetch each URL. Sight checks for both, and its llms.txt quality check looks for exactly this structure: an H1 title on the first line, a > summary, at least one ## section, and links that actually resolve.

Who actually reads llms.txt today?

Honestly: adoption is still early, on both sides. As of mid-2026 no major AI assistant has committed publicly to consuming llms.txt the way every serious crawler consumes robots.txt. It is a proposed convention, not a ratified standard, and you should treat any claim that a specific model "uses" it with some skepticism.

So why bother? Because the cost is close to zero and the downside is nil. Publishing one clean markdown file can't hurt your SEO, doesn't change how browsers or search crawlers see your site, and positions you for a standard that's plausibly heading toward wider support. It also forces a useful exercise: deciding, in plain language, what your site is and which ten pages you'd most want an AI to cite. If you can't write that summary easily, that's a content problem worth knowing about.

How does llms.txt differ from robots.txt and sitemap.xml?

They solve different problems and don't replace each other. robots.txt sets crawl permissions — who may fetch what. sitemap.xml is an exhaustive machine-readable list of every URL you want indexed. llms.txt is neither a permission file nor a complete inventory; it's an editorial summary aimed at models, curated down to what matters. You want all three: robots to govern access, a sitemap for coverage, and llms.txt for a human-quality introduction. For the crawl-permission side of that trio, see our guide on robots.txt and sitemaps.

Where to start

Write the file by hand — it's short. Name your site, summarize it in one sentence, and list the pages you'd most want quoted: your core product or service pages, your pricing, your best explainer content. Serve it as plain text, confirm it loads at /llms.txt in a browser, and check that every link in it resolves. That's the whole job.