What is llms.txt and do I need one?

TL;DR

llms.txt is a plain-text file at the root of your domain that gives AI crawlers and language models a curated summary of your site, what your business does, what your most citation-worthy pages are, and where to find them. It’s the AI-era equivalent of robots.txt for content discovery, and it’s cheap to ship.

The llms.txt convention emerged in 2024 from Jeremy Howard at fast.ai and was adopted by major AI tooling vendors and a growing number of documentation sites in 2025. The format is plain Markdown: a top-line summary of your site, followed by sections with links to your most important pages.

The file lives at https://yourdomain.com/llms.txt. AI crawlers and indexing systems read it to understand what your site is about without having to crawl every page first. Sites that ship a clean llms.txt get indexed faster and cited more accurately than sites that don’t.

For a small business, llms.txt does three things: it gives AI engines a one-sentence positioning statement they can quote when asked "what is [your company]," it lists your highest-signal pages (services, FAQ, answers) so the engines know what to fetch first, and it documents what you sell at what price so an engine answering a pricing query can cite a primary source.

Key facts

llms.txt was proposed by Jeremy Howard (fast.ai) in 2024 and gained adoption through 2025.
The file is plain Markdown, no special syntax, no XML, no JSON.
Recommended structure: top-line summary, then Markdown headings for each major section of your site, with bulleted links beneath.
Major adopters as of 2026 include Anthropic, Mintlify, Stripe documentation, and a wide range of developer-tool vendors.
Unlike robots.txt, llms.txt is a positive signal, what to read, not what to block.

Common follow-ups

Is llms.txt a standard?

It is a community convention rather than an IETF standard, but it has been adopted widely enough, including by tooling that AI engines themselves use, that treating it as a standard is reasonable in 2026.

How is llms.txt different from a sitemap?

A sitemap lists every URL on your site for crawlers to enumerate. llms.txt curates the small subset of pages a language model should read first to understand your business. Both are useful; they answer different questions.

How often should I update llms.txt?

Whenever you publish new high-signal content (a new service page, a new answer page, a pricing change). The file is small and the update is one commit; treat it as a living index.

Sources

By Isaiah Grant, Founder, Rebuilt StudioUpdated Apr 27, 2026