llms.txt: What It Is, How to Create One, and Whether It Actually Works

Somewhere in the last year, llms.txt went from a niche GitHub proposal to a checkbox on every AI SEO audit template. Tools flag you for not having one. Agencies sell its creation as a line item. And yet, if you ask the engines themselves whether they read it, the answer is silence. This page is the honest version: what llms.txt actually is, the exact format, how to build one in an hour, and what it can and cannot do for your generative engine optimization work.

What llms.txt is, in one paragraph

llms.txt is a proposed standard: a plain markdown file served at the root of your domain, at yoursite.com/llms.txt, that curates your most important content for large language models. It was proposed in September 2024 by Jeremy Howard, co-founder of Answer.AI (and before that, fast.ai), and the specification lives at llmstxt.org. The core idea is that LLMs operate with limited context windows and struggle with the noise of real web pages: navigation, cookie banners, footers, scripts. A curated markdown index gives an AI system a clean, prioritized reading list with just enough context to know what each page contains and which ones matter most.

That is the whole proposal. It is not a blocking mechanism, not a licensing declaration, and not a magic ranking file. It is a menu you write for machine readers.

The exact format

The spec is deliberately minimal. An llms.txt file contains, in order:

1. An H1 with the name of your site or project. This is the only required element.
2. A blockquote with a one-or-two-sentence summary of what the site is.
3. Optional free-form paragraphs with any context an LLM needs to interpret the rest.
4. H2 sections, each containing a markdown list of links in the format [Title](url): one-line description.
5. An optional section literally titled ## Optional, listing secondary URLs an LLM can skip when context is tight.

The choice of markdown is deliberate: it is both human-readable and trivially parseable by the models themselves, with no schema, no XML, no tooling required.

A realistic example for a small SaaS

Here is what a complete, honest llms.txt looks like for a small B2B SaaS. Adjust the sections to your own structure:

# Meeeters

> Meeeters is a link building platform: a non-reciprocal link
> network plus an AI article generator driven by a real site
> audit. It helps small businesses earn backlinks and publish
> structured content from one dashboard.

## Product

- [How it works](https://meeeters.com/how-it-works): the 3-way,
  non-reciprocal link exchange model explained step by step.
- [Free SEO analysis](https://meeeters.com/free-seo-analysis):
  audit any domain and get recommended pages and link targets.

## Guides

- [How to rank in AI search](https://meeeters.com/seo/how-to-rank-in-ai-search):
  pillar guide on how AI assistants retrieve and cite sources.
- [What is GEO](https://meeeters.com/geo/what-is-generative-engine-optimization):
  definition and scope of generative engine optimization.
- [Backlinks hub](https://meeeters.com/backlinks): everything on
  earning authority with relevant dofollow links.

## Optional

- [Blog index](https://meeeters.com/blog): all articles,
  including older posts.

Notice what makes this useful rather than decorative: every description says what a reader (human or machine) gets from the page, the list is short (curation is the point, a 400-line llms.txt defeats its own purpose), and the URLs are absolute.

llms-full.txt: the heavyweight variant

The spec also describes a companion convention: llms-full.txt, a single file containing the full text of your key content, not just links. Documentation-heavy projects use it so a model can ingest an entire docs site in one request. For a typical business site, llms-full.txt is usually overkill: it duplicates content you already serve as HTML, needs regeneration on every content change, and can balloon past any useful context size. If you maintain developer docs, generate it from your docs pipeline. If you run a marketing site and a blog, the standard llms.txt is enough.

llms.txt vs robots.txt vs sitemap.xml

These three files get conflated constantly, and they do completely different jobs:

Two practical takeaways. First, if your goal is controlling AI crawlers, that is a robots.txt job, not an llms.txt job: user agents like GPTBot respect robots.txt directives, and Google documents its crawlers in Google Search Central. Second, llms.txt does not replace your sitemap. Keep all three.

The honest adoption status

Here is the part most llms.txt articles bury: no major AI engine has publicly confirmed that it consumes llms.txt to build answers. Not OpenAI, not Google, not Perplexity, not Anthropic. Some site owners see AI-associated user agents request the file in their server logs, which proves curiosity at most, not influence. A fetched file and a used file are different things.

The most quotable skepticism came from Google. John Mueller, Google's Search Advocate, publicly compared llms.txt to the keywords meta tag: a self-declared signal that site owners control entirely, which is exactly why engines learned to ignore that class of signal years ago. The comparison stings because it is structurally accurate. Any file where a site describes its own importance, with no external verification, is trivially gameable, and engines have two decades of experience discounting gameable inputs.

Meanwhile, adoption on the publishing side is real but shallow: thousands of sites (especially developer tool companies) have shipped the file, and content platforms have added one-click generation. Publishing adoption without consumption adoption is a standard waiting for a customer.

So the honest framing is this: llms.txt today is a bet with a tiny stake. If the standard gets adopted, early publishers win a small head start. If it never does, you spent an hour. What it is not, under any current evidence, is a ranking lever. If a tool or agency presents llms.txt as the thing standing between you and ChatGPT citations, that tells you more about the tool than about your site.

Why we ship one anyway

Meeeters serves an llms.txt, and we recommend clients do too, for three reasons that hold even at zero engine adoption.

The cost is nearly zero. One markdown file, written once, updated when your key pages change. There is no plausible downside: it cannot hurt rankings, it cannot confuse crawlers, and it does not conflict with robots.txt or your sitemap.

Writing it is a useful exercise. Forcing yourself to pick the ten pages that best represent your site, and to describe each in one line, is a miniature content audit. Sites that struggle to fill an llms.txt usually have a real problem: no clear pillar pages, no obvious canonical answer for their main topics. That diagnosis is worth the hour by itself, and it feeds directly into a proper GEO audit.

Optionality is cheap right now. Standards sometimes tip quickly. If a major engine announces support, the sites with clean files already in place get whatever early benefit exists while everyone else scrambles.

That said, keep the effort proportional. An afternoon, not a sprint.

How to create your llms.txt, step by step

Step 1: Pick your pages. Choose 8 to 15 URLs maximum: your pillar guides, your product pages, your highest-value resources. If a page would not make your "best of" list for a human, it does not belong in llms.txt.

Step 2: Write the header. One H1 with your brand name, one blockquote that says what you do in plain language. Use the same one-line brand description you use everywhere else; entity consistency matters far more to AI systems than this file does, and mixed descriptions across the web are one reason ChatGPT may not know your brand.

Step 3: Group links into H2 sections. Two to four sections is plenty: Product, Guides, Docs, whatever matches your structure. Each link gets the format [Title](absolute-url): one-line description of what the page delivers.

Step 4: Add an Optional section if needed. Anything secondary (blog index, changelog, about page) goes here so a context-limited model knows it can skip it.

Step 5: Serve it at the root. The file must resolve at https://yoursite.com/llms.txt, served as plain text (UTF-8, text/plain or text/markdown both work in practice). On most stacks this means dropping the file in your public or static directory. On Next.js, put it in /public. On WordPress, upload it to the web root or use a plugin that serves virtual files.

Step 6: Verify and maintain. Open the URL in a browser, check every link resolves with a 200, and add a quarterly reminder to refresh it when key pages change. A stale llms.txt pointing at 404s is worse than none, because the one thing it was supposed to provide was reliability.

Common llms.txt mistakes

Since the file takes an hour, the mistakes are mostly mistakes of misunderstanding, not effort. The four we see most often in audits:

Treating it as a second sitemap. Exporting every URL on the domain into llms.txt misses the entire point of the format. A sitemap is exhaustive by design; llms.txt is selective by design. If a model has budget for ten of your pages, the file exists to say which ten. Past 20 or so entries, you have written a worse sitemap in the wrong syntax.

Marketing copy in the descriptions. "Discover our revolutionary approach to..." tells a machine nothing. Descriptions should be flatly informative: what the page contains, what question it answers. The model is deciding what to read, not whether to be impressed.

Blocking the file from the crawlers it targets. Genuinely common: a site ships llms.txt while its robots.txt, CDN or bot-protection layer blocks GPTBot and PerplexityBot outright. If your AI policy is to block, the manifest is pointless; if your policy is to be read, check that the readers can get in. Your server logs will tell you which AI user agents actually reach you.

Letting it rot. Product pages get renamed, guides get merged, and the llms.txt written in one enthusiastic afternoon quietly fills with 404s and outdated descriptions. The file claims to be your site's most reliable self-summary; a stale one is an argument for ignoring the standard. Tie its review to whatever quarterly content review you already run.

What to do instead of, and alongside, llms.txt

If llms.txt is a lottery ticket, the following are the paycheck. Everything we can observe about how assistants pick sources says they lean on retrieval from ranked, parseable, trustworthy pages, which the pillar on how to rank in AI search covers in depth. In priority order:

Clean, server-rendered HTML. AI crawlers are worse at JavaScript execution than Googlebot. If your content only exists after client-side rendering, some AI systems see an empty page. Semantic headings, real HTML tables and content that dominates the markup do more for machine readability than any manifest file.

Answer-first structure. Every important page should open its sections with the direct answer, then develop. Assistants extract passages; a page whose H2s each resolve one question cleanly is quotable, and a page that buries answers under wind-ups is not.

Structured data. Schema markup is the machine-readable layer that, unlike llms.txt, has confirmed consumers: Google's systems parse it today, and knowledge graphs feed the retrieval layers AI answers sit on. Organization, Article and FAQPage markup are the priorities; our guide to schema markup for AI search covers exactly which types matter and why.

Authority. This is the gate. AI visibility is downstream of search authority: assistants cite the handful of pages that already rank, and backlinks plus brand mentions decide who those are. A perfectly formatted llms.txt on a domain with no authority curates content nobody retrieves. This is the imbalance we built Meeeters around: the link network earns the authority while the content side keeps your pages structured and quotable, from one dashboard rather than two disconnected tools.

After you ship the file: where Meeeters picks up

llms.txt takes an hour; the inputs with receipts (rankings, structure, schema, links) take sustained work, and that is the part Meeeters automates from a single starting audit.

• The free SEO analysis crawls your site, detects the schema and JSON-LD you already serve, maps your silo structure, and surfaces the pillar pages worth listing in your llms.txt in the first place. No card required.
• Drafts generated from that audit fill your cluster gaps, arriving in your CMS as drafts (native Webflow connector, or webhook via Make, Zapier or n8n) so you keep full editorial control.
• The non-reciprocal three-way link network works the authority gate: dofollow links from real, vetted sites, one earned for each verified link you give.
• Google Search Console integration then shows whether it is working: clicks, impressions and the queries sitting just off page one.

Spend the afternoon on the file if you like, but run the free SEO analysis first: it tells you whether the manifest or the fundamentals deserve the hour.

The takeaway

llms.txt is a good idea with no confirmed adoption: a one-hour, zero-risk file that might matter later and definitely will not rescue an unloved site today. Ship one, keep it short and current, and spend the rest of the week on the inputs with receipts: rankings, structure, schema and links.

Christopher Fernandes

Founder of Meeeters

I built Meeeters to make link building safe and simple: real, relevant backlinks with no reciprocal footprint and no black-hat shortcuts. Questions about your site? Write to me directly.

Email us

We reply fast, usually within a few hours.