Contents
- The short version
- The words you will need
- Where the term came from
- How an agent reads your site
- What a person sees and what an agent sees
- Agentic website vs regular website
- Agentic website vs agentic web, AI site builders and chatbots
- Why the term took off now
- Who visits a site: the real list
- The second side: a site run with agents
- What is inside an agentic website: five layers
- Three types of agentic site
- An agentic website for B2B and fintech
- Security and control
- Myths and facts
- How to build an agentic website: 10 steps
- How to measure an agentic website
- Agentic website examples
- Agentic website checklist: score your site
- How we counted the site data
- Frequently asked questions
In two weeks of October, AI bots visited a small studio's site 11,022 times, and ChatGPT read its pages for real people 14 times a day. Here is what an agentic website is, how agents read it, how a team runs it with agents, and a 10-point checklist.
An agentic website is a site that AI agents work with from both sides: outside agents read it and act for people, and inside agents draft and update its pages while a person approves every change.
The term is young, and people use it in two senses. Most often it means an agent-ready website, one that AI agents can read and use. Less often it means a site that the team runs with the help of agents. We join both meanings here, because in practice the same team builds the two sides together.
For years, only people read websites. Now software takes over part of that reading: ChatGPT, Perplexity or Gemini find a company, compare offers and fill in a form for a person. A website for AI agents is written so that this software understands it from the first request. Inside, it works in a new way too: an agent builds a page in hours, and a marketer reviews and publishes it.
A shop shows the idea best. Personal shoppers have started to come in, buying for other people, reading price tags fast and choosing the store where everything is labelled. That is the first side. Behind the counter, meanwhile, helpers now stock the shelves by the chain's rules, while the manager checks the shelves and signs off. That is the second side. An agentic website is like a shop that is ready for both.

The short version
- The term has two sides. A site for agents that read and act, and a site with agents that prepare changes for a person to approve.
- An agent reads the page's code. Text has to arrive in the HTML from the server, and buttons and fields need names, because the major AI crawlers skip JavaScript.
- Agents are already among your visitors. Even a small studio's site had 11,022 AI bot visits in two weeks. ChatGPT read its pages for real people 14 times a day on average.
- Bots differ by purpose. Some train models, some build search, and some act for a person, so each type calls for its own decision.
- A person publishes. Agents prepare drafts and a person has the final word, which is what separates an agentic website from a site written by AI.
The words you will need
Ten terms come up later in the article. Here is each one in a single line.
| Term | What it means |
|---|---|
| AI agent | Software that does a task for a person: it searches, opens pages and fills in forms. Examples: ChatGPT agent, Gemini in Chrome, Perplexity Comet, Claude. Browsers with such agents built in are called agentic browsers. |
| Crawler (bot) | Software that goes through pages and reads them. It can serve search, model training or actions a person asks for. |
| HTML from the server | The page arrives with its text already in place, and the browser only has to show it. Agents read exactly this version. |
| robots.txt | A text file at the site's root with rules for bots: who may read which pages. |
| CDN and bot protection | A network of servers between the site and the visitor, Cloudflare for example. It speeds the site up and can stop bots, sometimes useful ones too. |
| Structured data | Schema.org markup: facts about the company, services, prices and FAQ in a form a machine reads reliably. |
| Markdown | A plain text format with minimal styling. Agents find it the easiest to read, so some sites serve markdown copies of their pages. |
| llms.txt | A file at the site's root that briefly tells a language model what the site is and where its main pages are. |
| Element labels | The names of buttons and form fields in the page code. Screen readers for blind people read them, and agents find their way by them too. |
| MCP | An open standard that connects an AI app to a service's data and actions, like a device to a power socket. |
Where the term came from
The agentic web took shape over two years, and each year added one important piece. In September 2024, Jeremy Howard proposed llms.txt, the first file written specifically for language models. In November 2024, Anthropic opened up MCP, a standard that gives AI access to data and actions.
In May 2025, at its Build conference, Microsoft named this picture "the open agentic web". In it, agents make decisions and perform tasks for people and companies. In July 2025, OpenAI launched ChatGPT agent, which opens websites and fills in forms by itself. In December 2025, MCP moved to the Linux Foundation, with more than 10,000 public MCP servers already running.
In September 2026, Cloudflare changed its default settings and began to block some agents on pages with ads. So agents went from an experiment to ordinary visitors, and every site owner now has to make decisions about them.
How an agent reads your site
On 13 October 2026, at 19:07 Kyiv time, ChatGPT opened a roundup of real estate websites on figmatica.com. At that minute someone was asking the assistant about developer websites, and it went to the studio's page for an answer. The question itself stays with the assistant, since it passes the site only the page address. The server log, however, records the time and the page to the second.
Here is such a visit, broken into steps. Say your buyer asks ChatGPT to find three suppliers and book a demo with the best one.
- The assistant searches its index. Pages get there when the site lets in the OAI-SearchBot crawler.
- It reads the pages it found on the person's behalf, as the ChatGPT-User bot. It receives only the HTML. Prices that a script draws in the browser stay a blank space for it, so it takes the figures from someone else's review.
- The assistant gathers prices, terms and case studies into a table. Facts written as text and structured data land there accurately, while a price inside an image gets lost.
- ChatGPT agent opens the demo form and finds its way by the names of fields and buttons, the way a screen reader does. It fills in a labelled field on the first try.
- Before sending, the agent asks the person: "Book for Thursday, 11:00?" The person agrees, and the request goes out.
A site that passes all five steps gets the lead. A site that breaks at any one of them drops out of the conversation, and its owner stays unaware of the loss.
The same site's log shows two more patterns. The assistant reads the home page most often. That is how it learns who the company is and what it does before naming it in an answer. Claude also opened robots.txt, the sitemap and llms.txt on their own several times. So it first checked what it was allowed to read and where the main content sits.

What a person sees and what an agent sees
A person sees the finished page: photos, animation, prices in neat cards. An agent sees raw HTML and a list of the page's elements with their names, the same list that screen readers use. Chrome's documentation for its Lighthouse tool calls this list the primary data model of a page for agents.
Live examples make the difference easy to see. We checked 12 bank websites with a plain request, the way a bot makes one. Three of them answered with an error or dropped the connection (our banking study). On these sites a person sees a handsome home page, while an agent gets an empty response.

Free check
What does ChatGPT see on your website?
Enter your site's address and see what an assistant reads from your home page: the text, the markup, llms.txt and whether OpenAI's crawler gets in.
The same six checks we ran on the 12 banks. The result appears in about ten seconds.
Agentic website vs regular website
Eight traits separate a regular website from an agentic one. The table puts them side by side.
| Regular website | Agentic website | |
|---|---|---|
| Who reads it | People | People, AI assistants and agents |
| How text reaches the page | Often through JavaScript in the browser | As finished HTML from the server |
| Facts | In paragraphs of text | As text and schema.org structured data |
| Forms and buttons | Names known only to people who see the screen | Every field and button has a name in the code |
| Bot access | A blanket block or the CDN's default settings | Named agents, let in on purpose |
| Updates | A ticket for a developer, days or weeks | An agent drafts, a person approves, hours |
| Brand | Each new page drifts a little from the style | Pages are built from the design system |
| Measurement | Visits in analytics | Visits plus mentions in AI answers |
Agentic website vs agentic web, AI site builders and chatbots
These four ideas often get mixed up, although each one describes a different scale.
| Concept | What it is | Example |
|---|---|---|
| Agentic web | The whole part of the internet where AI agents read and act for people. At Build 2025, Microsoft called it "the open agentic web". | ChatGPT, Gemini and Claude agents across the internet |
| Agentic website | Your site in that web: clear to agents and run with their help. | A company site with HTML from the server, labelled forms and approved changes |
| Agentic website builder | A tool that generates a site from a description. Its output becomes an agentic website once it has every trait from the table above. | A page generated from a prompt |
| Chatbot on a site | A widget in the corner that a visitor talks to. It covers one function. | An "Ask us" window |
Why the term took off now
Four figures explain the hurry. According to Cloudflare's annual review, AI bots made 4.2% of all requests to web pages in 2025. The share is still small, but in the same year, visits that assistants made at people's request grew more than fifteenfold.
Imperva gives the wider picture. In 2024, automated traffic overtook human traffic for the first time in a decade and reached 51%. Bots became the majority of internet visitors, and AI bots are the fastest-growing group among them.
The same growth shows up on a single site. In late September 2026, AI bots visited figmatica.com 184 times a day on average. In the first half of October, that figure was 787. Over the same period, ChatGPT visits made at people's request rose from 7 to 14 a day.
Agentic websites, meanwhile, are still rare. In April 2026, Cloudflare checked the 200,000 most popular domains, and only 4% of them set their own rules for AI. So whoever prepares their site now will meet the agents before their competitors do.

Who visits a site: the real list
To see agents live, take one working site, figmatica.com, and every AI bot that visited it from 1 to 14 October 2026. The server counts them, because Google Analytics runs through JavaScript and bots skip it.
| Bot | Owner | Type and purpose | Visits |
|---|---|---|---|
| Meta AI (meta-externalagent) | Meta | Learner: in Meta's words, training AI models and improving products | 9,678 |
| PerplexityBot | Perplexity | Searcher: the index for Perplexity's answers | 433 |
| Amazonbot | Amazon | Learner and searcher: in Amazon's words, better products and more accurate answers | 269 |
| ChatGPT-User | OpenAI | Helper: reads a page when a person asks ChatGPT | 194 |
| Bytespider | ByteDance | ByteDance's crawler; the company keeps its purpose to itself | 190 |
| OAI-SearchBot | OpenAI | Searcher: the index for ChatGPT search | 186 |
| Claude-User | Anthropic | Helper: reads a page when a person asks Claude | 41 |
| GPTBot | OpenAI | Learner: training OpenAI's models | 24 |
| DuckAssistBot | DuckDuckGo | Helper: reads pages in real time for DuckDuckGo's AI answers | 5 |
| ClaudeBot | Anthropic | Learner: training Anthropic's models | 1 |
| Total | 11,022 |
The loudest bot on the list matters least for leads. Meta made 88% of all visits, but it collects data for its own models. For a business, the 194 ChatGPT-User visits and the 41 Claude-User visits count most. Behind each one is a person who has just asked something.
That gives a rule for the three types. The learners are your call, and that decision has little effect on how assistants find you. Open the door to searchers and helpers, because a site closed to them disappears from answers.
Google barely appears on this list, because Google's AI Overviews read a site with the regular Googlebot. Bots also readily take markdown copies of pages: Meta AI opened them on figmatica.com whenever such copies were available.
The most common mistake site owners make is to close the door to every AI bot with one line in robots.txt. Along with the learners, the searchers then disappear too. The second trap hides in bot protection settings: since 15 September 2026, Cloudflare itself blocks some agents on pages with ads.

The second side: a site run with agents
The second half of the definition solves an old marketing problem. A campaign page waits weeks for a developer and goes live after the campaign has already ended.
On an agentic website, a change goes through five steps, and each step takes hours, so the days of waiting stay in the past.
- The marketer gives the agent a task: "A page for the 12 November webinar, for CFOs."
- The agent builds the page from ready-made blocks of the design system, so it matches the brand from the start.
- A person sees the draft and a list of changes in a separate copy of the site.
- The person edits the text or publishes the page.
- Analytics shows how the page performs from day one.
Large tools already work this way. In the Framer site builder, agents make changes in a separate copy of the site, and a person reviews and publishes them. Cursor rebuilt its own website so that every change an agent makes goes through review before it goes live.
This setup has a limit, and it pays to know it in advance. In a busy week, agents prepare more than a person can review. Gartner predicts that over 40% of agentic AI projects will be cancelled by the end of 2027. The reasons it names are costs, unclear value and weak risk control. So the approval step deserves the same care as the pages, since it is what keeps control in your hands.

What is inside an agentic website: five layers
Picture an agentic website as a five-storey building, where each floor rests on the one below it.
- Content: HTML from the server, structured data, facts as text and answers to real buyer questions.
- Access: a robots.txt with named agents, plus the CDN and bot protection settings.
- Action: labelled forms and buttons and a stable layout, so an agent can click and fill things in.
- Team workflow: a design system, an admin panel with roles and approvals, and agents that prepare drafts.
- Measurement: analytics, experiments and visibility in AI answers.

Three types of agentic site
Agentic websites differ by what the agent comes to do. On a company or service site, it reads and quotes prices, services and case studies, and most B2B, SaaS and fintech sites work this way. In a shop, the agent completes a purchase for a person, so the site gives it a clear path to checkout. A product or service site goes furthest and opens actions to agents directly, through MCP for example.
An agentic website for B2B and fintech
Most articles about the agentic web talk about shops, where an agent buys sneakers. In B2B, the agent comes for something else. It reads prices and plans, builds a comparison, checks case studies and fills in the demo form. So on a B2B site, plans written as text, comparison pages and a request form with labelled fields come first.
Fintech adds a regulatory requirement, because a lawyer or a compliance officer checks every change on the site. On an agentic website, the lawyer becomes one more approval step in the same process. Each change also keeps its history: who approved it and when.
Security and control
Agents bring a site new visitors, and they also give bot protection more work. Large agents such as ChatGPT agent sign their requests cryptographically, so protection can let them through and stop the rest.
Agents check with a person before any action with consequences. Gemini in Chrome, for example, pauses and asks for confirmation before a purchase or a post. Forms call for extra care, because labelled fields make the work easier for agents and spammers alike. So form protection sits on the server, with rate limits and email verification.
Myths and facts
Five common misconceptions have already grown up around the new term.
| Myth | Fact |
|---|---|
| AI writes an agentic website | Agents prepare drafts, and a person publishes |
| Adding a chatbot is enough | A chatbot covers one function, while an agentic website concerns the whole structure of the site |
| Google's AI Overviews need llms.txt | Google Search ignores llms.txt; Chrome Lighthouse checks for it, and the file takes an hour of work |
| Blocking every AI bot is the safest choice | A blocked OAI-SearchBot removes the site from ChatGPT search |
| Agents will come someday later | Assistant visits made at people's request grew fifteenfold in one year |
How to build an agentic website: 10 steps
The path to an agent-ready website starts with cheap steps and ends with more expensive ones. The first six fit into a week, while the last four reshape how the team works.
- Check what agents see today. The free site check shows the text available before any JavaScript runs, the markup, llms.txt and access for OpenAI's crawler.
- Serve text from the server: static generation or server-side rendering for pages with prices, services and case studies.
- Open the right doors: name the searchers and helpers one by one in robots.txt, and point to llms.txt at the bottom of the file. Check the CDN settings too.
- Add structured data that matches the visible text: organisation, services, FAQ.
- Label every field and every button. What helps people who use screen readers helps agents too.
- Publish llms.txt and markdown copies of your pages. It is cheap, and Lighthouse already checks for these files. How to write one, and what it really does: what is llms.txt.
- Answer buyer questions on the pages themselves. Our AEO guide explains how.
- Build a design system and an admin panel with approvals, so agents assemble pages from ready-made blocks. This is agentic website design, which we offer as a service.
- Let agents prepare drafts in a separate copy of the site, and leave publishing to a person.
- Measure the result, as the next section describes.
How to measure an agentic website
Google Analytics counts people, because it runs through JavaScript in the browser, and bots skip it. So an agentic website is measured in two places.
The first place is the server. It records every request whose user agent carries the name of a known AI bot, and it helps to keep two logs. The first log shows how many times each bot opened each page in a month. The second holds the visits made at a person's request, timed to the second. Requests from one assistant that arrive within 90 seconds of each other can then be grouped into one answer to one person.
The second place is analytics. In GA4, that means referrals from chatgpt.com, perplexity.ai, gemini.google.com and claude.ai, and above all, leads from the people who came from there.
Three more figures complete the picture. Once a month, check whether an assistant names you in its answers to 20 to 30 of your buyers' questions. The second figure tracks leads that arrived after a conversation with an assistant. The third counts the days from idea to published page, before agents and after them.
Agentic website examples
Three sites show different parts of the agentic approach.
Figmatica.com shows the visitor side. The site gives agents HTML from the server, llms.txt, a markdown copy of every page and named rules for 14 AI bots in robots.txt.
Cursor shows the code-based way to run a site. The site lives in code, and every change an agent makes goes through review as a pull request.
Framer shows the site-builder way to run a site. Agents make changes in a separate copy of the site, and a person reviews and publishes them.

Agentic website checklist: score your site
Count how many of these items get a "yes" from you.
- Text on pages with prices and services arrives in the HTML from the server
- robots.txt lets in search bots and bots that act at a person's request
- The CDN and bot protection settings are tested against ChatGPT
- Structured data matches the visible text
- Every form field and every button has a name
- llms.txt and markdown copies of pages are published
- Pages answer buyer questions
- New pages are built from the design system
- Agents prepare drafts, and a person approves them
- Visibility in AI answers is measured alongside analytics
Eight to ten yeses mean the site is already agentic. Five to seven mean agents read you, but competitors are taking part of the ground that could be yours. Four or fewer mean it is time to start with step 1, the site check.
Want a site agents can read and your team runs with agents?
In a free 30-minute call, we look at your site together and name the first three steps toward an agentic website.
Book a free call ↗How we counted the site data
- Site: figmatica.com, the website of one design studio, so a single case rather than a sample.
- Period: 1 to 14 October 2026 compared with 21 to 30 September 2026. Bot counting has run since 21 September, and the log of visits made at people's request since 25 September.
- Method: the server flags a request when its user agent carries the name of a known AI bot from a list of 17 names. A user agent can be faked, so the figures are an upper estimate.
- Excluded: requests from the studio's office network.
- Outside the count: Google's AI Overviews, because they read the site with the regular Googlebot, and the people's questions themselves.
Websites and products we shipped
BloxFi80+ country pages from one template: how a fintech API went global
Sona AIThe AI assistant that brings order to chaos
Acvian30+ pages in 60 days: a new Webflow site for a global HR partner
Want the same for your site? 30 minutes, free.
Book a free call ↗Sources
- Web crawlers (Meta-ExternalAgent), Meta, 2026
- The 2025 Cloudflare Radar Year in Review, Cloudflare, 2025
- Introducing the Agent Readiness score, Cloudflare, 2026
- Cloudflare allows the agentic internet to flourish with a simple philosophy: your content, your rules, Cloudflare, 2026
- The age of agents: cryptographically recognizing agent traffic, Cloudflare, 2025
- 2025 Imperva Bad Bot Report, Imperva, 2025
- Microsoft Build 2025: the age of AI agents and building the open agentic web, Microsoft, 2025
- AI features and your website, Google Search Central, 2025
- Chrome gets new Gemini 3 features, including auto browse, Google, 2026
- Lighthouse agentic browsing scoring, Chrome for Developers, 2026
- The rise of the AI crawler, Vercel, 2024
- Overview of OpenAI crawlers, OpenAI
- Does Anthropic crawl data from the web, and how can site owners block the crawler?, Anthropic
- Introducing the Model Context Protocol, Anthropic, 2024
- Donating MCP to the Agentic AI Foundation, Anthropic, 2025
- About Amazonbot, Amazon
- DuckAssistBot, DuckDuckGo
- Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027, Gartner, 2025
- The /llms.txt file, llmstxt.org (Jeremy Howard), 2024
- Introducing Framer Agents, Branching, and the new Community, Framer, 2026
- Coding Agents & Complexity Budgets, Lee Robinson, 2025




