llms.txt: What It Is, What Goes in It, and Whether It Actually Works

llms.txt is a proposed standard: a plain markdown file at the root of your website that gives AI systems a curated summary of what your site is and where its important content lives. Think of it as a sitemap written for language models — prose and links instead of XML. Interest in it has exploded, and so has the noise around it. Here’s the format, a working example, and a straight answer about what it does and doesn’t buy you in 2026.

Where it came from

The proposal was published in September 2024 by Jeremy Howard of Answer.AI, with the spec maintained at llmstxt.org. The reasoning: language models work within limited context windows and are terrible at wading through nav menus, cookie banners and 400KB of markup to find the substance of a site. A single curated markdown file solves that — the site tells the model what matters, in a format models parse natively.

Two files are defined: /llms.txt (the curated index) and optionally /llms-full.txt (full content of key pages inlined into one document, common for technical docs).

The format — four simple parts

An llms.txt is ordinary markdown with a required shape:

• An H1 with the site or project name — the only strictly required element.
• A blockquote summary — one or two sentences saying what the site is.
• Optional free paragraphs of context.
• H2 sections containing link lists — [Title](url): description per line. A final section named Optional marks links that can be skipped when an AI is short on room.

A real example for a small business

# Acme Roofing of Daytona Beach
> Family-owned roofing contractor serving Volusia County, Florida
> since 1998. Licensed & insured (CCC-1331234). Free estimates.

Acme installs and repairs shingle, tile and metal roofs for homes
and light-commercial buildings across the Daytona Beach area.

## Services
- [Roof Replacement](https://acme.example/replacement/): full
  tear-off and replacement, with warranty details and typical costs
- [Storm & Leak Repair](https://acme.example/repair/): emergency
  service, insurance-claim assistance

## Company
- [About Us](https://acme.example/about/): history, licensing,
  service area
- [Reviews](https://acme.example/reviews/): verified customer reviews

## Optional
- [Blog](https://acme.example/blog/): maintenance guides and
  hurricane-season prep

That’s the entire trick. Name, plain-language summary, and your most important pages with honest descriptions. Write it like you’re briefing a smart assistant who’s never seen your site — because that’s literally the use case.

Who actually supports it — the honest part

Here is where most articles oversell, so let’s be precise. Publishing llms.txt files is widespread: thousands of sites ship one, developer-documentation platforms like Mintlify and GitBook generate them automatically, and major tech companies’ docs sites carry them. Consuming them is the unproven half: none of the major AI providers — OpenAI, Anthropic, Google — has committed to reading llms.txt in their crawling or retrieval pipelines, and Google’s search folks have been openly skeptical, comparing it to the old meta-keywords tag.

So why do we still ship one on the sites we build? Three reasons, none of them hype:

• The cost is nearly zero. Fifteen minutes, no code, no risk, no downside for search — Google neither rewards nor penalizes it.
• Agents and tools read it today. A growing tail of AI agents, coding assistants and research tools fetch /llms.txt when pointed at a site, even though the big consumer assistants don’t officially. For those visitors, you control the narrative.
• It’s a cheap option on a possible future. If any major provider flips it on, sites with a good file are ahead by exactly one afternoon of work they did early.

Treat it as one small brick in the larger wall — the wall being actual generative engine optimization: crawler access, entity schema, content and reputation. An llms.txt on a site that blocks AI crawlers at the firewall is a welcome mat behind a locked door.

How to create yours

• By hand — it’s markdown; the example above is a complete template. Save it as llms.txt at your site root so it resolves at yourdomain.com/llms.txt.
• WordPress — several plugins now generate one from your pages and menus; check that the output reads like the curated example above and not like a raw page dump.
• Static sites — drop the file in your public directory; it deploys like any other asset. (On Cloudflare Pages, that’s the whole job.)

Keep it current the way you keep a sitemap current: revisit when your services or key pages change. And spend the description lines saying true, specific things — model-facing copy is still copy.

Frequently asked questions

Does Google use llms.txt?

No — Google has said its search and AI products don’t use it, and its search advocates have publicly compared it to the meta-keywords tag. Publishing one neither helps nor hurts your Google rankings.

What's the difference between llms.txt and robots.txt?

They’re opposites in spirit: robots.txt tells crawlers what they may not access; llms.txt tells AI systems what they should read first, with context. One is a fence, the other is a tour guide. Sites can and do have both.

Where does the llms.txt file go?

At your site’s root, so it resolves at https://yourdomain.com/llms.txt — the same location convention as robots.txt. The optional llms-full.txt lives at the root as well.

Is llms-full.txt worth doing too?

For documentation-heavy sites, yes — it inlines full page content so a model can ingest everything in one fetch. For a typical business site the curated llms.txt alone is plenty.