llms.txt in 2026: what it does, what Google says, whether to bother

The short answer

llms.txt is a plain Markdown file at /llms.txt that gives language models and AI agents a curated map of a site. Google says Google Search does not use it, and the two largest studies so far found no sign that having one changes how often a site is cited in AI answers. It is quick to make, so publish one if developers or agents read your documentation, and do not expect it to move search or AI visibility.

What is llms.txt?

llms.txt is a proposal Jeremy Howard published at llmstxt.org on 3 September 2024. It asks site owners to put a Markdown file at the root of a site that tells a language model which pages matter and where clean versions of them live.

The problem it targets is real. Web pages carry navigation, ads and scripts that are hard to turn into clean text, and the proposal points out that context windows are still too small to hold most websites whole. A short index plus Markdown copies of the key pages lets a model fetch only what it needs.

The proposal is aimed at inference, the moment a tool answers a question. Its authors say they expected the file to be useful mainly then, and less for model training.

It remains a community proposal. No standards body has adopted it, and no crawler is obliged to read it.

What goes in an llms.txt file?

The format is ordinary Markdown in a fixed order. Only the first element is required.

Part	Required	What it holds
H1 heading	Yes	The name of the site or project
Blockquote	No	A short summary
Free text	No	Further context, in any Markdown except headings
H2 sections of links	No	Lists of [name](url) links, each with an optional note after a colon
An H2 named “Optional”	No	Secondary links an agent can skip when it needs less context

A minimal file for a made-up product looks like this:

# Example Queue

> Example Queue is a hosted message queue. These pages cover setup, the API and limits.

## Docs

- [Quick start](https://example.com/docs/quick-start.md): install and send a first message
- [API reference](https://example.com/docs/api.md): every endpoint with request and response shapes

## Optional

- [Changelog](https://example.com/changelog.md)

The proposal also asks for a clean Markdown copy of each page at the same address with .md added (page.html.md) or swapped in for the extension (page.md).

What changed in version 2 of the proposal?

llmstxt.org now presents itself as version 2 of the proposal, last modified on 10 August 2026 and revised after two years of use. The v2 text covers Markdown versions of pages, discovery through link relations and scoping a file to one URL path. A page links to its Markdown version with rel="alternate" type="text/markdown" and to the llms.txt that covers it with rel="describedby", either as HTML <link> elements or as an HTTP Link header.

The proposal page also lists directories of published files and platforms that generate one for you, including Mintlify, GitBook, Yoast SEO, AIOSEO and Wix. A site on one of those can end up with an llms.txt without anyone deciding to publish it.

Does Google use llms.txt?

Google Search does not. Google has said so several times, and since May 2026 it has said so in its own documentation.

When	Who	What was said
April 2025	John Mueller, reported by Search Engine Journal	No AI service had said it uses the file, and server logs showed they do not even request it
July 2025	Gary Illyes at Search Central Live Deep Dive Asia Pacific, reported by Search Engine Land	Google does not support llms.txt and is not planning to crawl it
July 2025	John Mueller, reported by Search Engine Roundtable	It can make sense to noindex the file so people do not land on it from search
December 2025 to January 2026	Google’s documentation platform	An llms.txt appeared on the Search developer docs and the Search team removed it; Mueller said files left on other Google sites were no endorsement
15 May 2026	Google Search Central	The new guide to generative AI features says no new machine-readable files, AI text files, markup or Markdown are needed to appear in Google Search
15 June 2026	Google Search Central	The guide was clarified: Google Search does not use llms.txt, and keeping one for other services neither helps nor harms rankings

Mueller’s April 2025 comparison is the one most people remember: “To me, it’s comparable to the keywords meta tag”. The point was trust. A file in which a site describes itself adds nothing a search engine cannot check by reading the site.

Google’s general page on AI features says the same thing from the other direction. A page is eligible to appear as a supporting link in AI Overviews or AI Mode if it is indexed and eligible to show with a snippet, and there are no additional technical requirements.

Why does Chrome check for llms.txt then?

Because Chrome is looking at a different job. Lighthouse 13.2.0, released on 1 May 2026, added an llms.txt check inside a new agentic browsing category, and Lighthouse 13.3.0 on 7 May 2026 added that category to the default configuration.

The Chrome documentation calls llms.txt an emerging convention that gives a machine-readable summary of a site. The audit fails only when requesting the file produces a server error. A 404 is marked not applicable, because the file is optional.

So Google holds two positions that do not conflict. Search ranking and AI Overviews ignore the file. Chrome treats it as one optional signal that a site is easy for browser agents to work with.

Do ChatGPT, Claude or Perplexity read llms.txt?

None of them documents it as an input. The crawler pages published by OpenAI, Anthropic and Perplexity describe their bots and the robots.txt controls for each, and none of them says those bots use llms.txt to choose what to cite. According to llmstxt.org, OpenAI, Anthropic and Google’s Gemini team publish llms.txt files for their own developer docs, which is a different thing from reading yours.

Server logs give the clearest picture. Ahrefs looked at the 137,210 domains in Ahrefs Web Analytics that had traffic in May 2026 and published the results in June 2026:

• 28% of the domains published an llms.txt file.
• 97% of those files received no requests at all in May 2026.
• 96% of requests to llms.txt files came from bots.
• AI bots made 19.5% of requests. GPTBot, which OpenAI documents as its crawler for model training, was the most active named AI bot, and Claude-Code was second.
• SEO audit tools (21.7%), unidentified bots (14.9%) and general web crawlers (13.1%) each sent more requests than any single AI bot.

Ahrefs is careful about the limits. Its customers skew technical, so 28% is an upper bound on adoption, and a request does not prove a bot used the file.

A coding agent in second place fits the proposal’s own aim. Tools that work for a developer inside a codebase fetch documentation indexes. Systems deciding which sources to cite in a public answer show little sign of doing so.

How many sites have one, and do they get cited more?

Adoption is modest and the citation effect is not visible in the data.

	SE Ranking	Ahrefs
Published	November 2025	June 2026
Sample	About 300,000 domains	137,210 domains on Ahrefs Web Analytics
Share with llms.txt	10.13%	28%, an upper bound
What was measured	Link between the file and AI citation frequency	Requests to /llms.txt in May 2026
Finding	No clear effect on citations	97% of files received no requests

SE Ranking used Spearman correlation, an XGBoost model and SHAP analysis. Removing the llms.txt variable made its model more accurate on test data, and the authors concluded the file may currently add more noise than value. Adoption was lower among the highest-traffic sites: 8.27% for domains with more than 100,000 visits, against 9.88% for those with 100 or fewer.

Neither study is a controlled experiment, and neither can rule out a small effect on one particular engine. Together they give no support for treating llms.txt as a lever worth paying for.

Can llms.txt do harm?

Not to rankings, according to Google. The practical risks are smaller and self-inflicted:

• A file that lists moved or deleted pages sends agents to dead ends. Generate it from the same source as your sitemap so it cannot drift.
• Markdown copies of pages duplicate your HTML. Keep them out of the index or point them at the HTML page with a canonical link.
• A route that returns a server error for /llms.txt fails the Lighthouse audit. A clean 404 does not.
• The file itself can be indexed if other sites link to it, which is why Mueller suggested a noindex header.

When is it worth publishing one?

It depends on who reads your site.

Site	Worth it?	Why
Developer docs, APIs, SDKs	Yes	Coding agents fetch documentation indexes, and docs platforms generate the file for free
Product with a help centre	Probably	A clean index helps agents answer support questions with the right page
Publisher, blog or local business	Optional	No search or citation effect has been shown; time is better spent on original, indexable pages

What to do

1. Get the documented basics right first: pages crawlable, indexed and eligible for a snippet. That is what AI Overviews and AI Mode draw on.
2. If you publish developer documentation, add llms.txt and Markdown page copies, generated from your docs build so they stay current.
3. For other sites, accept the file if your CMS or SEO plugin creates it, and do not spend more than an hour on it.
4. Make sure /llms.txt returns either a 200 with Markdown or a clean 404, never a server error.
5. Send an X-Robots-Tag: noindex header on the file and keep Markdown copies out of the index.
6. Check your server logs for requests to /llms.txt after a month. If nothing reads it, there is nothing more to optimise.
7. Treat any service that sells llms.txt as a way to get cited in AI answers with suspicion. No public evidence supports that claim.