llms.txt Checker: How to Validate Your File and Format

An llms.txt checker confirms that /llms.txt exists on your domain, returns HTTP 200 with raw markdown, starts with an H1, and lists real, working links. It cannot tell you whether any AI product will use the file, because no major vendor has committed to reading it.

What llms.txt is, briefly

The proposal, published at llmstxt.org, asks sites to serve a markdown file at the root that summarises what the site is and links to its most useful pages. BusinessMCP’s llms.txt guide describes the format as minimal and parsed top to bottom:

• An H1 with the site or project name, the only strictly required element.
• A blockquote summary of what the site is.
• Optional free text, without headings.
• H2 sections, each a list of links with title, URL and an optional one-line description after a colon.
• An optional H2 named “Optional” for secondary material.

A companion proposal, llms-full.txt, holds the full text of key content in a single file.

Be realistic about the benefit

The same guide says there is no evidence it is a ranking factor and calls adoption uneven. Its argument for publishing is that the cost is low and risk near zero. That is a fair framing: treat llms.txt as cheap hygiene, not a growth lever. It complements robots.txt rather than replacing it, since robots.txt controls access while llms.txt aims at comprehension.

Key takeaway: a valid llms.txt proves you published a file. It does not prove anyone reads it. Check it for correctness, and spend your effort where the evidence is stronger.

What a checker should verify

Test	Pass condition	Why it matters
File exists	/llms.txt returns 200	Missing file earns nothing
Content type	Plain text or markdown, not an HTML page	Soft-404 pages often return 200 with HTML
Starts with H1	First line is # Name	The only required element
Summary present	A > blockquote follows	Gives models context
Link sections	H2 headings containing link lists	The useful part
Links resolve	Each URL returns 200	Stale links mislead
Reachable by crawlers	Not blocked by CDN or WAF rules	An invisible file is useless
Size sane	Curated, not a sitemap dump	Long lists defeat the purpose

The “soft 404” row catches many sites: a catch-all route serves the homepage at every path, including /llms.txt, so a naive existence test passes. Always inspect the body, not just the status code.

A manual validation routine

1. Run curl -i https://yourdomain.com/llms.txt and read the status and Content-Type headers.
2. Confirm the first non-empty line starts with # .
3. Confirm every list item is a markdown link in the form - [Title](URL): description.
4. Extract all URLs and request each; flag anything that is not 200.
5. Repeat with a crawler-like user agent to see whether your CDN treats it differently.
6. Check llms-full.txt if you publish one.

What the AI readiness grade does with it

BusinessMCP’s grader awards 15 of 100 points for the presence of an llms.txt file, and fetches llms.txt and llms-full.txt as part of its checks. Presence is what is scored, per the grader’s description, not quality. So a file that passes a presence check can still be badly formed; the AI readiness score article explains how the other 85 points break down.

If you want to draft a file rather than just validate one, BusinessMCP also offers a free llms.txt generator. Review any generated output by hand: a good file is curated, with an honest one-line description per link.

Writing a file worth checking

The guide’s steps are sensible:

1. Select the pages that matter, such as core docs, best guides, pricing and policies. Keep the list curated.
2. Write the H1 and a short summary of the business.
3. Add H2 sections with a link and an honest line for each.
4. Add an “Optional” section for secondary material.
5. Serve it at the root as raw markdown with a 200 status.
6. Make sure crawlers can reach it.
7. Update it as you ship pages.

Automating the check

You can script every row of the table above in an afternoon. If you would rather not, the readiness grade through the website checks API covers presence as part of one $0.03 call, and you can add your own link-resolution step on top. For a portfolio, fold it into your bulk domain audit.

Conventions like this move fast. Check the proposal site and the vendors’ documentation for the current state before you invest more than an hour in it.

Example structure to compare against

A well-formed small file looks like this in outline: an H1 with the site name, a one-line blockquote describing the site, a short paragraph of context, then H2 sections such as “Guides” and “Reference”, each with three to eight links and a few words of description per link, and an “Optional” section at the end. That is the whole format. If your file has dozens of sections or hundreds of links, you have rebuilt a sitemap, which defeats the purpose that BusinessMCP’s guide describes.

Common defect	How a checker spots it	Fix
HTML returned instead of markdown	Content-Type or body begins with a tag	Serve the raw file, not a templated page
Missing H1	First line does not start with #	Add the site name as the first line
Relative links	Links lack a scheme and host	Use absolute URLs
Dead links	Non-200 on request	Update or remove them
Blocked by firewall	403 for bot user agents	Allow the path for crawlers

Related checks that matter more

Because the evidence for llms.txt is thin, balance your effort. Crawler access, structured data and clear page content are better-understood foundations. WebMCP is another emerging convention with its own caveats. Treating each as a low-cost experiment, with a recorded date and a way to detect breakage, is a sensible stance for all of them.

Monitoring the file

Files like this vanish in deploys. A scheduled check that confirms status 200, a first-line H1 and a stable link count is cheap insurance. BusinessMCP lists checking llms.txt from a scheduled job as a use for its API, and our guide to scheduled website monitoring shows how to set it up with change-only alerts.