llms.txt: What It Is, What Goes in It, and Whether It Actually Works

llms.txt is a proposed standard: a plain markdown file at the root of your website that gives AI systems a curated summary of what your site is and where its important content lives. Think of it as a sitemap written for language models — prose and links instead of XML. This guide covers the format, a fictional example and the evidence to consider before adding one.

Where it came from

The proposal was published in September 2024 by Jeremy Howard of Answer.AI, with the spec maintained at llmstxt.org. The reasoning: language models work within limited context windows and are terrible at wading through nav menus, cookie banners and 400KB of markup to find the substance of a site. A single curated markdown file solves that — the site tells the model what matters, in a format models parse natively.

Two files are defined: /llms.txt (the curated index) and optionally /llms-full.txt (full content of key pages inlined into one document, common for technical docs).

The format — four simple parts

An llms.txt is ordinary markdown with a required shape:

• An H1 with the site or project name — the only strictly required element.
• A blockquote summary — one or two sentences saying what the site is.
• Optional free paragraphs of context.
• H2 sections containing link lists — [Title](url): description per line. A final section named Optional marks links that can be skipped when an AI is short on room.

Example for a fictional small business

# Acme Roofing of Daytona Beach
> Family-owned roofing contractor serving Volusia County, Florida
> since 1998. Licensed & insured (CCC-1331234). Free estimates.

Acme installs and repairs shingle, tile and metal roofs for homes
and light-commercial buildings across the Daytona Beach area.

## Services
- [Roof Replacement](https://acme.example/replacement/): full
  tear-off and replacement, with warranty details and typical costs
- [Storm & Leak Repair](https://acme.example/repair/): emergency
  service, insurance-claim assistance

## Company
- [About Us](https://acme.example/about/): history, licensing,
  service area
- [Reviews](https://acme.example/reviews/): verified customer reviews

## Optional
- [Blog](https://acme.example/blog/): maintenance guides and
  hurricane-season prep

That’s the entire trick. Name, plain-language summary, and your most important pages with honest descriptions. Write it like you’re briefing a smart assistant who’s never seen your site — because that’s literally the use case.

Publishing and use are different

Publishing a file makes it available; it does not prove that a tool uses it. Consumption needs separate evidence from the tools relevant to your audience. Google’s published guidance says Google Search, including its generative AI features, does not use llms.txt.

We include the file as an optional publishing aid. Here is how to assess it:

• Allow for upkeep. The file is simple to create, but review it for accuracy and keep it current. Treat it as optional until you have evidence that relevant tools use it.
• Check the tools you use. If a tool supports /llms.txt, the file can provide an organized starting point.
• Revisit its usefulness. Keep the file if it serves your publishing workflow, and reassess its value as tools change.

Treat it as one small brick in the larger wall — the wall being actual generative engine optimization: crawler access, entity schema, content and reputation. An llms.txt on a site that blocks AI crawlers at the firewall is a welcome mat behind a locked door.

How to create yours

• By hand — it’s markdown; the example above is a complete template. Save it as llms.txt at your site root so it resolves at yourdomain.com/llms.txt.
• WordPress — several plugins now generate one from your pages and menus; check that the output reads like the curated example above and not like a raw page dump.
• Static sites — drop the file in your public directory; it deploys like any other asset. (On Cloudflare Pages, that’s the whole job.)

Keep it current the way you keep a sitemap current: revisit when your services or key pages change. And spend the description lines saying true, specific things — model-facing copy is still copy.

Frequently asked questions

Does Google use llms.txt?

Google says llms.txt is not used for Google Search, including its generative AI features. It is optional for other tools that use such files.

What's the difference between llms.txt and robots.txt?

They’re opposites in spirit: robots.txt tells crawlers what they may not access; llms.txt tells AI systems what they should read first, with context. One is a fence, the other is a tour guide. Sites can and do have both.

Where does the llms.txt file go?

At your site’s root, so it resolves at https://yourdomain.com/llms.txt — the same location convention as robots.txt. The optional llms-full.txt lives at the root as well.

Is llms-full.txt worth doing too?

For documentation-heavy sites, yes — it inlines full page content so a model can ingest everything in one fetch. For a typical business site the curated llms.txt alone is plenty.