llms.txt

llms.txt is a proposed markdown file at a site's root that tells a language model what the site is and which pages to read first. It was proposed by Jeremy Howard on 2024-09-03. It is not a standard, and nothing enforces it.

The format is short: an H1 with the site name, an optional blockquote summary, optional markdown sections, then H2 sections holding lists of links shaped [name](url): note. An Optional heading has a defined meaning, which is that those links may be skipped when a shorter context is needed.

What the evidence says

This is where most pages about llms.txt stop being useful, because most of them are selling it.

• Google says Search does not use it. That is the company that would have to read it for the SEO claim to work.
• Ahrefs measured 97% of 137,000 llms.txt files receiving zero requests. Publishing one and getting no traffic is the normal outcome, not a failure of yours.
• Adoption is low but not zero. On 2026-08-03, 4.76% of the 2,228 domains Domduck read that day served one. The live figure moves slowly.

The honest model is that the file gets read when an agent is pointed at your domain, not when a crawler happens by. That is worth something if your audience runs coding agents, and worth nothing otherwise.

The half of the proposal everybody skips

Serve a clean markdown copy of each page at the same URL with .md appended. Stripe and Cloudflare both do it in production, and it is the part that changes what a model actually receives: a page without navigation, a theme toggle and a footer.

Domduck does the same. /docs/mcp has a markdown twin at /docs/mcp.md, and /llms.txt points at the markdown copies rather than the HTML.

What Domduck records

Presence, daily, on the same pass that reads robots.txt. One boolean with a date, and the corpus figure above comes from it.

Contents are not stored and not parsed. A file whose format is a proposal rather than a specification would need a parser with opinions, and opinions in a measurement are how a study stops being reproducible.

Should you publish one

Publish it if writing it costs you an hour and you can generate it from something. Do not publish it expecting rankings, and do not put access rules in it, because nothing enforces them. The control is robots.txt, and even that is advisory.

Generate it rather than hand-writing it. Every URL in the file has to resolve, and a hand-written list rots on the first rename with nothing to catch it. Domduck's own file is built from the documentation registry and a test asserts every link in it answers.