Contents
  1. The short version
  2. The words you will need
  3. Where the term came from
  4. How an agent reads your site
  5. What a person sees and what an agent sees
  6. Agentic website vs regular website
  7. Agentic website vs agentic web, AI site builders and chatbots
  8. Why the term took off now
  9. Who visits a site: the real list
  10. The second side: a site run with agents
  11. What is inside an agentic website: five layers
  12. Three types of agentic site
  13. An agentic website for B2B and fintech
  14. Security and control
  15. Myths and facts
  16. How to build an agentic website: 10 steps
  17. How to measure an agentic website
  18. Agentic website examples
  19. Agentic website checklist: score your site
  20. How we counted the site data
  21. Frequently asked questions

In two weeks of October, AI bots visited a small studio's site 11,022 times, and ChatGPT read its pages for real people 14 times a day. Here is what an agentic website is, how agents read it, how a team runs it with agents, and a 10-point checklist.

An agentic website is a site that AI agents work with from both sides: outside agents read it and act for people, and inside agents draft and update its pages while a person approves every change.

The term is young, and people use it in two senses. Most often it means an agent-ready website, one that AI agents can read and use. Less often it means a site that the team runs with the help of agents. We join both meanings here, because in practice the same team builds the two sides together.

For years, only people read websites. Now software takes over part of that reading: ChatGPT, Perplexity or Gemini find a company, compare offers and fill in a form for a person. A website for AI agents is written so that this software understands it from the first request. Inside, it works in a new way too: an agent builds a page in hours, and a marketer reviews and publishes it.

A shop shows the idea best. Personal shoppers have started to come in, buying for other people, reading price tags fast and choosing the store where everything is labelled. That is the first side. Behind the counter, meanwhile, helpers now stock the shelves by the chain's rules, while the manager checks the shelves and signs off. That is the second side. An agentic website is like a shop that is ready for both.

Diagram of an agentic website's two sides: visiting AI agents read, compare and act; editing agents draft pages a person approves
Outside, agents read and act. Inside, agents draft and a person approves.

The short version

  • The term has two sides. A site for agents that read and act, and a site with agents that prepare changes for a person to approve.
  • An agent reads the page's code. Text has to arrive in the HTML from the server, and buttons and fields need names, because the major AI crawlers skip JavaScript.
  • Agents are already among your visitors. Even a small studio's site had 11,022 AI bot visits in two weeks. ChatGPT read its pages for real people 14 times a day on average.
  • Bots differ by purpose. Some train models, some build search, and some act for a person, so each type calls for its own decision.
  • A person publishes. Agents prepare drafts and a person has the final word, which is what separates an agentic website from a site written by AI.

The words you will need

Ten terms come up later in the article. Here is each one in a single line.

TermWhat it means
AI agentSoftware that does a task for a person: it searches, opens pages and fills in forms. Examples: ChatGPT agent, Gemini in Chrome, Perplexity Comet, Claude. Browsers with such agents built in are called agentic browsers.
Crawler (bot)Software that goes through pages and reads them. It can serve search, model training or actions a person asks for.
HTML from the serverThe page arrives with its text already in place, and the browser only has to show it. Agents read exactly this version.
robots.txtA text file at the site's root with rules for bots: who may read which pages.
CDN and bot protectionA network of servers between the site and the visitor, Cloudflare for example. It speeds the site up and can stop bots, sometimes useful ones too.
Structured dataSchema.org markup: facts about the company, services, prices and FAQ in a form a machine reads reliably.
MarkdownA plain text format with minimal styling. Agents find it the easiest to read, so some sites serve markdown copies of their pages.
llms.txtA file at the site's root that briefly tells a language model what the site is and where its main pages are.
Element labelsThe names of buttons and form fields in the page code. Screen readers for blind people read them, and agents find their way by them too.
MCPAn open standard that connects an AI app to a service's data and actions, like a device to a power socket.

Where the term came from

The agentic web took shape over two years, and each year added one important piece. In September 2024, Jeremy Howard proposed llms.txt, the first file written specifically for language models. In November 2024, Anthropic opened up MCP, a standard that gives AI access to data and actions.

In May 2025, at its Build conference, Microsoft named this picture "the open agentic web". In it, agents make decisions and perform tasks for people and companies. In July 2025, OpenAI launched ChatGPT agent, which opens websites and fills in forms by itself. In December 2025, MCP moved to the Linux Foundation, with more than 10,000 public MCP servers already running.

In September 2026, Cloudflare changed its default settings and began to block some agents on pages with ads. So agents went from an experiment to ordinary visitors, and every site owner now has to make decisions about them.

How an agent reads your site

On 13 October 2026, at 19:07 Kyiv time, ChatGPT opened a roundup of real estate websites on figmatica.com. At that minute someone was asking the assistant about developer websites, and it went to the studio's page for an answer. The question itself stays with the assistant, since it passes the site only the page address. The server log, however, records the time and the page to the second.

Here is such a visit, broken into steps. Say your buyer asks ChatGPT to find three suppliers and book a demo with the best one.

  • The assistant searches its index. Pages get there when the site lets in the OAI-SearchBot crawler.
  • It reads the pages it found on the person's behalf, as the ChatGPT-User bot. It receives only the HTML. Prices that a script draws in the browser stay a blank space for it, so it takes the figures from someone else's review.
  • The assistant gathers prices, terms and case studies into a table. Facts written as text and structured data land there accurately, while a price inside an image gets lost.
  • ChatGPT agent opens the demo form and finds its way by the names of fields and buttons, the way a screen reader does. It fills in a labelled field on the first try.
  • Before sending, the agent asks the person: "Book for Thursday, 11:00?" The person agrees, and the request goes out.

A site that passes all five steps gets the lead. A site that breaks at any one of them drops out of the conversation, and its owner stays unaware of the loss.

The same site's log shows two more patterns. The assistant reads the home page most often. That is how it learns who the company is and what it does before naming it in an answer. Claude also opened robots.txt, the sitemap and llms.txt on their own several times. So it first checked what it was allowed to read and where the main content sits.

Five steps of an AI assistant on a website: search, read, compare, act, confirm
Each step can break: a closed bot, a price drawn by a script, a field with no name.

What a person sees and what an agent sees

A person sees the finished page: photos, animation, prices in neat cards. An agent sees raw HTML and a list of the page's elements with their names, the same list that screen readers use. Chrome's documentation for its Lighthouse tool calls this list the primary data model of a page for agents.

Live examples make the difference easy to see. We checked 12 bank websites with a plain request, the way a bot makes one. Three of them answered with an error or dropped the connection (our banking study). On these sites a person sees a handsome home page, while an agent gets an empty response.

ubank home page in a browser next to the 403 Access Denied response the same address returns to ChatGPT-User
The same address: a home page for the person, an error page for the assistant.

Free check

What does ChatGPT see on your website?

Enter your site's address and see what an assistant reads from your home page: the text, the markup, llms.txt and whether OpenAI's crawler gets in.

The same six checks we ran on the 12 banks. The result appears in about ten seconds.

Agentic website vs regular website

Eight traits separate a regular website from an agentic one. The table puts them side by side.

Regular websiteAgentic website
Who reads itPeoplePeople, AI assistants and agents
How text reaches the pageOften through JavaScript in the browserAs finished HTML from the server
FactsIn paragraphs of textAs text and schema.org structured data
Forms and buttonsNames known only to people who see the screenEvery field and button has a name in the code
Bot accessA blanket block or the CDN's default settingsNamed agents, let in on purpose
UpdatesA ticket for a developer, days or weeksAn agent drafts, a person approves, hours
BrandEach new page drifts a little from the stylePages are built from the design system
MeasurementVisits in analyticsVisits plus mentions in AI answers

Agentic website vs agentic web, AI site builders and chatbots

These four ideas often get mixed up, although each one describes a different scale.

ConceptWhat it isExample
Agentic webThe whole part of the internet where AI agents read and act for people. At Build 2025, Microsoft called it "the open agentic web".ChatGPT, Gemini and Claude agents across the internet
Agentic websiteYour site in that web: clear to agents and run with their help.A company site with HTML from the server, labelled forms and approved changes
Agentic website builderA tool that generates a site from a description. Its output becomes an agentic website once it has every trait from the table above.A page generated from a prompt
Chatbot on a siteA widget in the corner that a visitor talks to. It covers one function.An "Ask us" window

Why the term took off now

Four figures explain the hurry. According to Cloudflare's annual review, AI bots made 4.2% of all requests to web pages in 2025. The share is still small, but in the same year, visits that assistants made at people's request grew more than fifteenfold.

Imperva gives the wider picture. In 2024, automated traffic overtook human traffic for the first time in a decade and reached 51%. Bots became the majority of internet visitors, and AI bots are the fastest-growing group among them.

The same growth shows up on a single site. In late September 2026, AI bots visited figmatica.com 184 times a day on average. In the first half of October, that figure was 787. Over the same period, ChatGPT visits made at people's request rose from 7 to 14 a day.

Agentic websites, meanwhile, are still rare. In April 2026, Cloudflare checked the 200,000 most popular domains, and only 4% of them set their own rules for AI. So whoever prepares their site now will meet the agents before their competitors do.

Three industry figures, 4.2%, 15 times and 51%, and a chart of AI bot visits per day on figmatica.com, September to October 2026
Small share, fast growth: one site saw AI bot visits rise about four times in a month.

Who visits a site: the real list

To see agents live, take one working site, figmatica.com, and every AI bot that visited it from 1 to 14 October 2026. The server counts them, because Google Analytics runs through JavaScript and bots skip it.

BotOwnerType and purposeVisits
Meta AI (meta-externalagent)MetaLearner: in Meta's words, training AI models and improving products9,678
PerplexityBotPerplexitySearcher: the index for Perplexity's answers433
AmazonbotAmazonLearner and searcher: in Amazon's words, better products and more accurate answers269
ChatGPT-UserOpenAIHelper: reads a page when a person asks ChatGPT194
BytespiderByteDanceByteDance's crawler; the company keeps its purpose to itself190
OAI-SearchBotOpenAISearcher: the index for ChatGPT search186
Claude-UserAnthropicHelper: reads a page when a person asks Claude41
GPTBotOpenAILearner: training OpenAI's models24
DuckAssistBotDuckDuckGoHelper: reads pages in real time for DuckDuckGo's AI answers5
ClaudeBotAnthropicLearner: training Anthropic's models1
Total11,022

The loudest bot on the list matters least for leads. Meta made 88% of all visits, but it collects data for its own models. For a business, the 194 ChatGPT-User visits and the 41 Claude-User visits count most. Behind each one is a person who has just asked something.

That gives a rule for the three types. The learners are your call, and that decision has little effect on how assistants find you. Open the door to searchers and helpers, because a site closed to them disappears from answers.

Google barely appears on this list, because Google's AI Overviews read a site with the regular Googlebot. Bots also readily take markdown copies of pages: Meta AI opened them on figmatica.com whenever such copies were available.

The most common mistake site owners make is to close the door to every AI bot with one line in robots.txt. Along with the learners, the searchers then disappear too. The second trap hides in bot protection settings: since 15 September 2026, Cloudflare itself blocks some agents on pages with ads.

Table of ten AI bots that visited figmatica.com from 1 to 14 October 2026, by owner, type and number of visits
Learners in grey, searchers in blue, helpers in pink. 11,022 visits in two weeks.

The second side: a site run with agents

The second half of the definition solves an old marketing problem. A campaign page waits weeks for a developer and goes live after the campaign has already ended.

On an agentic website, a change goes through five steps, and each step takes hours, so the days of waiting stay in the past.

  • The marketer gives the agent a task: "A page for the 12 November webinar, for CFOs."
  • The agent builds the page from ready-made blocks of the design system, so it matches the brand from the start.
  • A person sees the draft and a list of changes in a separate copy of the site.
  • The person edits the text or publishes the page.
  • Analytics shows how the page performs from day one.

Large tools already work this way. In the Framer site builder, agents make changes in a separate copy of the site, and a person reviews and publishes them. Cursor rebuilt its own website so that every change an agent makes goes through review before it goes live.

This setup has a limit, and it pays to know it in advance. In a busy week, agents prepare more than a person can review. Gartner predicts that over 40% of agentic AI projects will be cancelled by the end of 2027. The reasons it names are costs, unclear value and weak risk control. So the approval step deserves the same care as the pages, since it is what keeps control in your hands.

The path of a change on an agentic website: task, agent draft, review, approval, live
Hours from brief to page, with a person at the gate.

What is inside an agentic website: five layers

Picture an agentic website as a five-storey building, where each floor rests on the one below it.

  • Content: HTML from the server, structured data, facts as text and answers to real buyer questions.
  • Access: a robots.txt with named agents, plus the CDN and bot protection settings.
  • Action: labelled forms and buttons and a stable layout, so an agent can click and fill things in.
  • Team workflow: a design system, an admin panel with roles and approvals, and agents that prepare drafts.
  • Measurement: analytics, experiments and visibility in AI answers.
Five layers of an agentic website as floors: content, access, action, team workflow, measurement
Blue floors make the site readable for agents, pink floors let the team run it with agents.

Three types of agentic site

Agentic websites differ by what the agent comes to do. On a company or service site, it reads and quotes prices, services and case studies, and most B2B, SaaS and fintech sites work this way. In a shop, the agent completes a purchase for a person, so the site gives it a clear path to checkout. A product or service site goes furthest and opens actions to agents directly, through MCP for example.

An agentic website for B2B and fintech

Most articles about the agentic web talk about shops, where an agent buys sneakers. In B2B, the agent comes for something else. It reads prices and plans, builds a comparison, checks case studies and fills in the demo form. So on a B2B site, plans written as text, comparison pages and a request form with labelled fields come first.

Fintech adds a regulatory requirement, because a lawyer or a compliance officer checks every change on the site. On an agentic website, the lawyer becomes one more approval step in the same process. Each change also keeps its history: who approved it and when.

Security and control

Agents bring a site new visitors, and they also give bot protection more work. Large agents such as ChatGPT agent sign their requests cryptographically, so protection can let them through and stop the rest.

Agents check with a person before any action with consequences. Gemini in Chrome, for example, pauses and asks for confirmation before a purchase or a post. Forms call for extra care, because labelled fields make the work easier for agents and spammers alike. So form protection sits on the server, with rate limits and email verification.

Myths and facts

Five common misconceptions have already grown up around the new term.

MythFact
AI writes an agentic websiteAgents prepare drafts, and a person publishes
Adding a chatbot is enoughA chatbot covers one function, while an agentic website concerns the whole structure of the site
Google's AI Overviews need llms.txtGoogle Search ignores llms.txt; Chrome Lighthouse checks for it, and the file takes an hour of work
Blocking every AI bot is the safest choiceA blocked OAI-SearchBot removes the site from ChatGPT search
Agents will come someday laterAssistant visits made at people's request grew fifteenfold in one year

How to build an agentic website: 10 steps

The path to an agent-ready website starts with cheap steps and ends with more expensive ones. The first six fit into a week, while the last four reshape how the team works.

  • Check what agents see today. The free site check shows the text available before any JavaScript runs, the markup, llms.txt and access for OpenAI's crawler.
  • Serve text from the server: static generation or server-side rendering for pages with prices, services and case studies.
  • Open the right doors: name the searchers and helpers one by one in robots.txt, and point to llms.txt at the bottom of the file. Check the CDN settings too.
  • Add structured data that matches the visible text: organisation, services, FAQ.
  • Label every field and every button. What helps people who use screen readers helps agents too.
  • Publish llms.txt and markdown copies of your pages. It is cheap, and Lighthouse already checks for these files. How to write one, and what it really does: what is llms.txt.
  • Answer buyer questions on the pages themselves. Our AEO guide explains how.
  • Build a design system and an admin panel with approvals, so agents assemble pages from ready-made blocks. This is agentic website design, which we offer as a service.
  • Let agents prepare drafts in a separate copy of the site, and leave publishing to a person.
  • Measure the result, as the next section describes.

How to measure an agentic website

Google Analytics counts people, because it runs through JavaScript in the browser, and bots skip it. So an agentic website is measured in two places.

The first place is the server. It records every request whose user agent carries the name of a known AI bot, and it helps to keep two logs. The first log shows how many times each bot opened each page in a month. The second holds the visits made at a person's request, timed to the second. Requests from one assistant that arrive within 90 seconds of each other can then be grouped into one answer to one person.

The second place is analytics. In GA4, that means referrals from chatgpt.com, perplexity.ai, gemini.google.com and claude.ai, and above all, leads from the people who came from there.

Three more figures complete the picture. Once a month, check whether an assistant names you in its answers to 20 to 30 of your buyers' questions. The second figure tracks leads that arrived after a conversation with an assistant. The third counts the days from idea to published page, before agents and after them.

Agentic website examples

Three sites show different parts of the agentic approach.

Figmatica.com shows the visitor side. The site gives agents HTML from the server, llms.txt, a markdown copy of every page and named rules for 14 AI bots in robots.txt.

Cursor shows the code-based way to run a site. The site lives in code, and every change an agent makes goes through review as a pull request.

Framer shows the site-builder way to run a site. Agents make changes in a separate copy of the site, and a person reviews and publishes them.

The same page on figmatica.com as a person sees it, as a Markdown copy for agents, and the site's llms.txt
One page, three versions: for people, for agents, for the whole site.

Agentic website checklist: score your site

Count how many of these items get a "yes" from you.

  • Text on pages with prices and services arrives in the HTML from the server
  • robots.txt lets in search bots and bots that act at a person's request
  • The CDN and bot protection settings are tested against ChatGPT
  • Structured data matches the visible text
  • Every form field and every button has a name
  • llms.txt and markdown copies of pages are published
  • Pages answer buyer questions
  • New pages are built from the design system
  • Agents prepare drafts, and a person approves them
  • Visibility in AI answers is measured alongside analytics

Eight to ten yeses mean the site is already agentic. Five to seven mean agents read you, but competitors are taking part of the ground that could be yours. Four or fewer mean it is time to start with step 1, the site check.

Want a site agents can read and your team runs with agents?

In a free 30-minute call, we look at your site together and name the first three steps toward an agentic website.

Book a free call ↗

How we counted the site data

  • Site: figmatica.com, the website of one design studio, so a single case rather than a sample.
  • Period: 1 to 14 October 2026 compared with 21 to 30 September 2026. Bot counting has run since 21 September, and the log of visits made at people's request since 25 September.
  • Method: the server flags a request when its user agent carries the name of a known AI bot from a list of 17 names. A user agent can be faked, so the figures are an upper estimate.
  • Excluded: requests from the studio's office network.
  • Outside the count: Google's AI Overviews, because they read the site with the regular Googlebot, and the people's questions themselves.

Websites and products we shipped

Want the same for your site? 30 minutes, free.

Book a free call ↗

Sources

  1. Web crawlers (Meta-ExternalAgent), Meta, 2026
  2. The 2025 Cloudflare Radar Year in Review, Cloudflare, 2025
  3. Introducing the Agent Readiness score, Cloudflare, 2026
  4. Cloudflare allows the agentic internet to flourish with a simple philosophy: your content, your rules, Cloudflare, 2026
  5. The age of agents: cryptographically recognizing agent traffic, Cloudflare, 2025
  6. 2025 Imperva Bad Bot Report, Imperva, 2025
  7. Microsoft Build 2025: the age of AI agents and building the open agentic web, Microsoft, 2025
  8. AI features and your website, Google Search Central, 2025
  9. Chrome gets new Gemini 3 features, including auto browse, Google, 2026
  10. Lighthouse agentic browsing scoring, Chrome for Developers, 2026
  11. The rise of the AI crawler, Vercel, 2024
  12. Overview of OpenAI crawlers, OpenAI
  13. Does Anthropic crawl data from the web, and how can site owners block the crawler?, Anthropic
  14. Introducing the Model Context Protocol, Anthropic, 2024
  15. Donating MCP to the Agentic AI Foundation, Anthropic, 2025
  16. About Amazonbot, Amazon
  17. DuckAssistBot, DuckDuckGo
  18. Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027, Gartner, 2025
  19. The /llms.txt file, llmstxt.org (Jeremy Howard), 2024
  20. Introducing Framer Agents, Branching, and the new Community, Framer, 2026
  21. Coding Agents & Complexity Budgets, Lee Robinson, 2025

Where we can help