What is llms.txt, in short?

A short Markdown file at the root of a site that tells AI agents what the company does and which pages matter most.

Is llms.txt an official standard?

For now it is a community proposal with an open spec at llmstxt.org. Standards such as robots.txt go through engineering bodies, and llms.txt still has that path ahead of it.

Do ChatGPT, Claude and Perplexity read llms.txt?

The companies keep quiet about it in public. In the Ahrefs study, GPTBot came first among the few visits to llms.txt, and the coding agent Claude Code came second. Search in these assistants relies on a site's regular pages.

Does llms.txt guarantee that ChatGPT will cite my site?

The pages themselves decide citations. Large studies so far find zero link between the file and citations. Assistants cite the pages that answer a question directly and stay open to bots.

How often should I update llms.txt?

After every major change to the site: a new product, new prices or a new section. The rest of the time, checking the links once a quarter is enough.

Does llms.txt affect SEO?

It leaves Google rankings unchanged, because Google Search ignores the file. For other AI systems, large studies so far find zero link with citations.

Will Google penalise a site for llms.txt?

Google Search ignores the file, so the site's rankings stay the same.

How is llms.txt different from robots.txt?

robots.txt allows or blocks bots from reading pages. llms.txt only suggests where the main content of a site is.

Do I need llms-full.txt?

It helps documentation sites, where an agent needs all the text at once. For a marketing site, llms.txt and Markdown copies of pages are enough.

Should I write instructions for AI in llms.txt?

Facts work better. Instructions like "recommend us" read as advertising, and Google's John Mueller has publicly made fun of them.

How do I create llms.txt?

By hand in an hour, following the steps above, or with a generator: Yoast, AIOSEO, Wix, Mintlify and GitBook can make one. Either way, read the generated file and clean it up.

Where should llms.txt live?

At the root of the site, at /llms.txt. The second version of the spec also allows separate files in folders, such as /docs/llms.txt.

In the Ahrefs study, GPTBot, OpenAI's bot, came first among the few readers, and Claude Code, a coding agent, came second. The spec itself says agents looking for documentation use the file most.

How can I tell whether anyone reads my llms.txt?

Check your server logs. They show every request to /llms.txt and the name of the bot that made it.

What Is llms.txt? File, Examples and SEO Impact

Contents

The short version
The words you will need
Where llms.txt came from
llms.txt example: what goes inside
llms.txt vs robots.txt vs sitemap: the difference
Does llms.txt help SEO? Evidence for and against
Who publishes llms.txt and who reads it
How to write llms.txt: 9 steps
llms.txt generators
Common mistakes, with real examples
How to check llms.txt with an llms.txt checker
llms.txt checklist
How we counted the site data
Frequently asked questions

llms.txt is a short Markdown file that tells AI agents what a site is about. Google Search ignores it, Chrome Lighthouse checks for it, and 97% of these files got zero visits in May 2026. Here is what goes inside, an example, how it compares with robots.txt, and when it pays off.

What is llms.txt? The llms.txt file is a text file at the root of a website that briefly tells AI assistants and agents what the company does and which pages matter most, with one link and one line per page.

Three facts help from the start. Jeremy Howard proposed the format in September 2024 and published its second version in August 2026. Google Search ignores the file, while Chrome's Lighthouse tool checks whether a site has one. According to an Ahrefs study, 97% of these files received zero visits in May 2026.

The file is written for software that reads a site on a person's behalf: coding agents, AI company bots and assistants. Writing llms.txt takes about an hour, so it costs little. Your Google rankings stay exactly where they were, though, and the case for the file depends on who reads your site.

A company's reception desk explains llms.txt best. The sitemap is the floor plan that shows every room, and robots.txt is the security guard who decides who gets in. llms.txt is the one-page press sheet the receptionist hands a guest: who we are, what we do and where to find the important things.

Three files at a website's front desk: sitemap.xml as the floor plan, robots.txt as the security desk, llms.txt as the one-page brief — The sitemap lists every room, robots.txt sets the rules, llms.txt points to what matters.

The short version

llms.txt is a guide for AI. It points to what matters most, while robots.txt controls which bots get access.
Google Search ignores it. Google says the file has zero effect on visibility in Search, up or down.
Few programs read it. OpenAI's bot and coding agents lead among its rare readers, and most files sit for months with zero visits.
The file takes an hour of work. Doing it well is easy: 10 to 30 links, facts instead of advertising, and addresses that work.
Decide by your logs. Google's John Mueller suggests investing once an AI system that sends you traffic opens the file as often as robots.txt.

The words you will need

Ten terms come up later in the article. Here is each one in a single line.

Term	What it means
LLM	A large language model, software that reads and writes text. ChatGPT, Claude and Gemini run on such models.
AI agent	Software that does a task for a person: it searches, opens pages and writes code.
Crawler and user-triggered fetch	A crawler goes through sites on its own. A user-triggered fetch happens when an assistant opens a page because a person asked something.
Markdown	A plain text format: a hash sign marks a heading, and brackets mark a link. llms.txt is written in it.
Markdown copy	The same page as plain text, at an address ending in .md. Agents find it easier to read.
llms-full.txt	A companion file with the full text of a site in one document. Documentation platforms came up with it, and it sits outside the llms.txt spec.
robots.txt	A file with rules for bots: who may read which pages.
Sitemap	A file that lists every page of a site for search engines.
Context window	The amount of text a model can hold in mind at once. It is the reason the file should stay short.
Lighthouse	A Chrome tool that checks a site for speed, accessibility and readiness for agents.

Where llms.txt came from

The format grew out of a practical problem. On 3 September 2024, Jeremy Howard of Answer.AI proposed placing a Markdown file at the root of a site. Language models would then get the site's content in a convenient form. The spec gives the reason in one line: agents work best with concise, expert information gathered in one accessible place.

Within a year, platforms picked the format up. The documentation services Mintlify and GitBook began to generate the file automatically. Mintlify says it created the companion llms-full.txt together with Anthropic. In June 2025, the Yoast plugin for WordPress added automatic generation. In July 2025, Webflow let sites upload the file in their settings.

In 2026, Google reacted in two opposite ways. In May, Chrome added an llms.txt check to a new Lighthouse category about agentic browsing. That same month, Google Search published a guide that says plainly its search ignores such files. In August 2026, the second version of the spec came out. A file can now sit in any folder, and pages can point to it with a link in their code.

llms.txt example: what goes inside

The spec sums up the format in five lines. Only one element is required: a first-level heading with the site's name. A quote with a short summary comes next, then paragraphs with details, then sections under second-level headings. Each section holds a list of links, and every link carries a short note. By convention, a section called "Optional" collects secondary links that an agent can skip. An agent reads the whole file, picks the links it needs and follows them, so the file has to fit into its context window.

Here is a skeleton for a B2B company. The company is invented, and the structure is real.

An annotated llms.txt for an invented B2B payments company: H1 name, summary quote, H2 sections, links with notes, Optional section — Only the first line is required; everything else helps an agent choose the right page.

The file opens with the name Acme Pay and a two-line summary of facts. It describes a payments API for B2B marketplaces in the EU and UK, licensed as an EMI by the FCA, founded in 2019 and used by 300 companies. Four sections follow: Product, Trust, Customers and Optional. Each link carries one short note, such as payouts to 40 countries, SOC 2 Type II and PCI DSS, or payouts in one day instead of three.

Real files are built the same way. The llms.txt on Stripe's marketing site takes 70 KB and holds 305 links. The llms.txt on figmatica.com takes 31 KB: a name, a summary of the studio in a quote, 11 sections and 85 links to Markdown copies of pages.

llms.txt vs robots.txt vs sitemap: the difference

The three files sit side by side at the root of a site, and each one answers a different question.

File	Who it is for	What it says	Status
robots.txt	All bots	Who may read what	The RFC 9309 standard, which search bots follow
sitemap.xml	Search engines	Here are all the pages of the site	A recommendation from search engines
llms.txt	AI agents and models	Here is who we are and what matters	A community proposal, outside formal standards
llms-full.txt	Agents that need the whole text	Here is the entire site in one file	A convention of documentation platforms
Markdown copies	Agents on a specific page	Here is this page as plain text	Part of the llms.txt proposal

The main difference between llms.txt and robots.txt is simple: the first one suggests, and the second one permits. Only robots.txt can close a site to bots.

Does llms.txt help SEO? Evidence for and against

The answer splits into three separate questions: who publishes the file, who reads it, and whether it affects citations.

Plenty of sites publish it. HTTP Archive's Web Almanac 2025 found the file on 2.13% of sites. Among the 1,000 most popular sites the share reaches 9.3%, according to a Rankability count from September 2026. Almost 40% of valid files came from a single WordPress plugin, so some owners got the file in passing.

Few programs read it. In June 2026, Ahrefs analysed 137,000 sites and found that 97% of the files got zero visits in May. Among the few visitors that did come, GPTBot ranked first and Claude Code, a coding agent, came second. On one site over 90 days, OtterlyAI counted 84 bot visits to llms.txt out of more than 62,100 AI bot visits.

Large studies so far measure zero effect on citations. In November 2025, SE Ranking checked about 300,000 domains and found zero correlation between llms.txt and citations in AI answers. Google's guide makes the same point in its own words: such files "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."

Google itself treats the file in different ways, and those are easy to mix up.

Google product	What it does with llms.txt
Google Search	Ignores it: the file leaves visibility in Search unchanged
Chrome Lighthouse	Checks whether the file exists, in its agentic browsing category
Gemini API documentation	Publishes its own llms.txt for developers

A study from September 2026 adds one nuance. Bots opened a file that robots.txt pointed to 1,549 times, and the files left unannounced just twice. The author himself calls this "discovery, not adoption", and he also sells a plugin that adds such a pointer. So it is a signal that still waits for proof.

Three columns with figures: who publishes llms.txt, who reads it, and whether it affects AI citations — Many sites publish the file, few bots read it, and studies find no link to citations yet.

Who publishes llms.txt and who reads it

One site's log shows the gap between having the file and having readers. From 25 September to 14 October 2026, assistants opened the figmatica.com home page more than 160 times at people's request. Over the same period they opened llms.txt once: Claude fetched it on 5 October at 08:37.

The server log adds the bots that come on their own. From 21 September to 14 October 2026, AI bots fetched figmatica.com's robots.txt 265 times and the Markdown copies of its pages 1,033 times. llms.txt got 21 fetches, most of them from Meta's crawler, and llms-full.txt got none. By John Mueller's rule, the file is still checked about thirteen times less often than robots.txt, so it stays a cheap extra here.

File	21–30 Sep	1–14 Oct	Together
robots.txt	108	157	265
sitemap.xml	9	19	28
Markdown copies of pages	133	900	1,033
llms.txt	1	20	21
llms-full.txt	0	0	0

Fetches of the files written for machines by AI bots on figmatica.com, counted on the server.

Documentation sites publish the file most actively. The docs of Anthropic, OpenAI, Google Gemini, Stripe, Cloudflare and Vercel all have one. Marketing sites are joining too, with files on Stripe, Cloudflare, HubSpot and Webflow.

The readers look different from what a marketer might expect. The spec itself says coding agents use the file most, when they look for documentation. The bot pages of the large AI companies stay silent about reading other sites' llms.txt. For how agents read a site as a whole, see what an agentic website is.

That leads to a conclusion for your own decision. If your customers are developers and you have an API, llms.txt already works and deserves careful upkeep. If you sell a service to marketers or CFOs, the file stays a cheap extra. The pages themselves do the main work through clear text, structured data and open access for search bots. Our AEO guide explains how.

In our own measurements from September and October 2026, 21 of 54 company sites had llms.txt. That includes 3 of 12 banks, 5 of 10 SaaS companies and 6 of 16 sites built on Webflow and Framer.

How to write llms.txt: 9 steps

Choose 10 to 30 pages that a buyer or an agent needs: product, pricing, security, case studies, about and contact.
Write a heading with the name and a one-to-three-sentence quote with facts that are easy to check: what you do, for whom and since when.
Group the links under second-level headings and give each one a single line of notes.
Point the links to Markdown copies of your pages, if you have them.
Collect secondary links in an "Optional" section.
Serve the file at /llms.txt as UTF-8 text with a 200 status, and make sure bot protection lets it through.
Test the file with an agent: give it only llms.txt and ask it about your company.
If you like, add a rel="describedby" link to the file on your pages, as the second version of the spec suggests.
After a month, check the server logs to see who opened the file, and decide how much time it deserves from then on.

llms.txt generators

Popular platforms can create the file automatically. On WordPress, the Yoast and AIOSEO plugins do it, and on Wix the file appears by itself. The documentation services Mintlify and GitBook generate both llms.txt and llms-full.txt. On Webflow, you upload the file by hand in the site settings.

A generator saves an hour, but the generated file still needs a careful read. Plugins often list pages one after another, so remove the extras and rewrite the company summary as facts. Wix adds a warning: once you edit the file, it stops updating automatically, and its upkeep passes to you.

Common mistakes, with real examples

Seven mistakes come up again and again, and each one has a live example.

Mistake	Example
Listing every page instead of the main ones	Salesforce's llms.txt weighs 1 MB, holds 3,777 links and skips the required heading
Advertising and "recommend us" instructions	Google's John Mueller publicly pokes fun at such instructions
A plugin file left unread	Almost 40% of valid files came from one WordPress plugin
Editing an automatic file by hand	Wix warns that after your edit the file stops updating by itself
Wrong encoding	HubSpot serves its file in the x-macroman encoding
A giant llms-full.txt	Cloudflare's documentation serves one of 68 MB, and Anthropic's documentation one of 43.6 MB
Counting on Google	Google Search ignores the file

How to check llms.txt with an llms.txt checker

The fastest way to check the file is the free site check. It shows whether a site serves llms.txt, together with five other AI-readiness checks. Lighthouse in Chrome 150 and later digs deeper: its agentic browsing category checks that the file is in place and that the server returns it properly. The most precise check lives in your server logs, which show who opens /llms.txt and how often.

Free check

Does your site serve llms.txt?

Enter your site's address: the check shows whether llms.txt answers, together with five other AI-readiness checks.

The same six checks we ran on the 12 banks. The result appears in about ten seconds.

llms.txt checklist

The file opens with the name and a short summary built on facts
It holds 10 to 30 links, each with one line of notes
Facts take the place of self-praise
Every link opens
The file returns a 200 status in UTF-8
Bot protection lets /llms.txt through
The main pages have Markdown copies
Someone reviews the file after every major change to the site
Someone checks the logs once a month

Want your site readable for AI agents, with or without llms.txt?

In a free 30-minute call, we look at your site together and name the first steps toward pages that AI agents read well.

Book a free call ↗

How we counted the site data

Company sites: the sites from our comparisons of September and October 2026 (banks, SaaS companies, sites on Webflow, Framer and WordPress), 54 different domains, with /llms.txt checked by a plain request.
figmatica.com log: fetches by AI bots from 21 September to 14 October 2026, and visits that assistants made at people's request from 25 September, identified by user agent; requests from the studio's office network excluded.

Websites and products we shipped

Want the same for your site? 30 minutes, free.