llms.txt checker
Check whether a site publishes an llms.txt file, and whether the file is actually valid.
For example: · No account, no email, no daily limit.
Also free, no account:AI crawler checkerDomain Rating checkerCertificate expiry checkerAll tools
What llms.txt is
A markdown file at the root of a site that tells a language model what the site is and where its useful pages are. The format is short: one H1 title, an optional summary in a blockquote, then sections of markdown links.
It is a proposal, not a standard. No search engine or model provider is obliged to read it, and most do not.
Does llms.txt do anything?
Google has said Search does not use llms.txt. Ahrefs looked at 137,000 published files and found 97% of them had never been requested. We sampled 216 domains on 3 August 2026 with our own parser and found the file on 12.6% of well-known sites and on none of the 69 domains we sampled from the middle of the Tranco list.
So it is a developer-tools convention rather than a web one. Publish one because your documentation is easier to read that way, not because it will move a ranking. Anybody telling you otherwise has not measured it.
What this checker looks at
It fetches the file and checks the structure the format describes: one H1 and only one, a summary blockquote, sections with links in them, and whether the URLs are absolute. It reports errors for things the format requires, warnings for things a reader will notice, and notes for things that are allowed and worth knowing.
It does not invent rules. There is no required section, no required link count and no schema, so a validator that demanded any of those would be checking against itself.
Why some checks are stored and some are not
Whether a domain serves an llms.txt at all is recorded every day, on the same request that reads its robots.txt. That has a history, and you can see the day it appeared or disappeared.
Whether the file is well formed runs live and is not stored. It is the answer to “did I write this correctly”, it is checked by the person who can fix it, and it would be worth nothing as a series.
llms-full.txt is not part of the format
The spec at llmstxt.org describes one file. The larger variant came from FastHTML, which generated llms-ctx-full.txt, and the name spread from there in several spellings. We probe for it and report whether it is there. We do not treat its absence as a fault.
llms.txt and robots.txt answer different questions
robots.txt says which crawlers may fetch what. llms.txt says what is worth reading. One is permission and the other is a table of contents, so publishing an llms.txt while blocking every AI crawler is not a contradiction, it is just unusual.
Our AI crawler checker answers the first question for eight named agents, in three states rather than two.
Every free tool here
- Domain Rating checker
Type a domain. See its Domain Rating, its popularity rank, how old it is, when its certificate expires and which AI crawlers it blocks.
- Bulk Domain Rating checker
Paste your list, one domain per line. No account, no captcha, no daily limit.
- AI crawler checker
Check what a site tells GPTBot, ClaudeBot, PerplexityBot and five more, and whether the edge agrees with the file.
- Certificate expiry checker
Check when a certificate expires, who issued it, and how many names it covers.
- Domain badges
Five embeddable badges for any domain, each linking to that domain’s own record.