Request a diagnostic

Agent-ready web

What makes a website usable by both people and software agents?

Direct answer

A website is agent-ready when the qualities that make it accessible to people also make it legible to software acting on someone's behalf: semantic HTML, a truthful accessibility tree, explicit labels and states, discoverable content, bounded and confirmable actions. Agent readiness is a quality layer on the same site, not a parallel machine version.

Last reviewed 2026-07-14Published 2026-07-14

The accessibility overlap

Agents read websites the way assistive technology does: through structure, names, roles and states. Semantic HTML, a truthful accessibility tree, explicit labels and visible state changes serve a screen-reader user and a browser agent identically. Sites that treat accessibility as compliance debt discover they have also been illegible to the software their customers are starting to send.

Discoverable, stable, explicit

Content must exist without JavaScript execution where possible, at stable URLs, with real text rather than text-in-images. Interfaces must be stable across visits: an agent that learned the form last week should find it this week. Controls need accessible names, and states need announcing in more than color.

Structured entities, sources and dates

An agent evaluating a business needs the same things a careful human does: who this is, what it offers, what supports the claims, when the material was last true. Structured data that mirrors visible content, source lists in the reading experience, and dated pages answer those questions machine-readably without creating a second, divergent version of the site.

Bounded actions and confirmation

Actions (forms, scheduling, purchases) must be labeled, bounded and confirmable. No hidden commitments, no hover-only paths, explicit confirmation before anything irreversible. This protects human users first and makes agent-completed actions safe second; the design work is the same work.

What llms.txt can and cannot do

llms.txt is a proposed convenience index, not a visibility mechanism: no major platform has confirmed consuming it. This site publishes one anyway, accurately and framed as an experiment, and logs whether anything fetches it. Crawler controls that demonstrably do things are robots.txt tokens: OAI-SearchBot governs ChatGPT search inclusion, GPTBot governs training use, Google-Extended governs Gemini training without touching Search. Each is a separate decision, documented on the Agent Index.

The Read and Inspect pattern

This site’s answer to machine legibility is one factual source rendered two ways. The article you are reading is canonical; the Inspect panel below exposes its entities, claims, sources, dates and relationships, derived from the same content objects. The same substance, structurally explicit.

Failure patterns

Content only in JavaScript or animation

Crawlers and agents receive an empty or misleading page

Hover-only controls and unlabeled buttons

Neither assistive tech nor agents can find or complete the action

A separate machine version

Hidden machine-targeted copy diverges from what people see; trust and policy risk

llms.txt as strategy

A convenience file mistaken for a visibility mechanism

Claims on this page

  • FactHigh confidenceChecked 2026-07-14

    OpenAI documents three separate bots, each independently controllable in robots.txt: OAI-SearchBot for ChatGPT search inclusion, GPTBot for model training, and ChatGPT-User for user-initiated fetches.

    Blocking GPTBot opts content out of training without removing it from ChatGPT search; blocking OAI-SearchBot removes search inclusion. Search inclusion and training are separate decisions.

    OpenAI

  • FactHigh confidenceChecked 2026-07-14

    Google-Extended is a robots.txt token controlling whether content is used to train future Gemini models and for grounding; Google states it does not affect inclusion in Google Search and is not a ranking signal.

    Googlebot honors the token; there is no separate crawler behind it. It does not control appearance in AI Overviews or AI Mode, which follow Search indexing.

    Google Search Central

  • FactHigh confidenceChecked 2026-07-14

    llms.txt is a September 2024 proposal by Jeremy Howard (Answer.AI) for an LLM-friendly markdown index; no major AI platform has confirmed using it, and Google staff have said no AI system currently consumes it.

    Publishing an llms.txt is low-cost and may be a useful convenience, but it is not a visibility mechanism and should not be sold as one.

    llmstxt.org (Answer.AI) · Search Engine Roundtable

  • ForecastLow confidenceHorizon: 24 months from 2026-07Checked 2026-07-14

    Agent-mediated visits and actions will remain a small share of most businesses' traffic over the next two years, but will be disproportionately high-intent where they occur.

    Agents visit with a task. The forecast is about share and character of traffic, not about which agent platforms win.

    OpenAI

  • InterpretationHigh confidenceChecked 2026-07-14

    Websites now function as inputs to machine-generated answers as well as destinations for clicks.

    Answer engines retrieve, interpret and cite page content. What a site makes legible (entities, claims, dates, sources, actions) shapes how it is represented in answers it never renders.

    OpenAI · Google Ads Help

Sources

  1. Bots — OpenAI crawler documentation OpenAI · published 2026-07-14 · accessed 2026-07-14 · primary

    Undated live documentation; date shown is the access date. Documents OAI-SearchBot (ChatGPT search inclusion), GPTBot (training) and ChatGPT-User (user-initiated fetches) as independently controllable.

  2. Google's common crawlers Google Search Central · published 2026-07-14 · accessed 2026-07-14 · primary

    Undated live documentation; date shown is the access date. Confirms Google-Extended "does not impact a site's inclusion in Google Search nor is it used as a ranking signal."

  3. The /llms.txt file llmstxt.org (Answer.AI) · Jeremy Howard · published 2024-09-03 · accessed 2026-07-14 · primary

    The proposal itself. No engine has confirmed consuming it.

  4. Google Says No AI System Currently Uses LLMs.txt Search Engine Roundtable · published 2025 · accessed 2026-07-14 · secondary

    Reports John Mueller's public statement.

Inspectstructure, entities, claims, sources, dates

Direct answer

A website is agent-ready when the qualities that make it accessible to people also make it legible to software acting on someone's behalf. It is a quality layer on the same site.

Page purpose

Define agent readiness, document crawler controls, and link the practices this site itself demonstrates.

Entities

  • Digital Traction
  • OAI-SearchBot
  • GPTBot
  • Google-Extended
  • llms.txt
  • Agent Index

Defined terms

  • Agent-ready website
  • Accessibility tree
  • llms.txt

Relationships

  • Depends on: Brand evidence, Content systems
  • Tested in: Agent website audit, llms.txt fetch log, Read vs Inspect retrieval
  • Demonstrated by: this site (/agent)

Dates and status

Published
2026-07-14
Last material update
2026-07-14
Last reviewed
2026-07-14
Status
published

Available actions

  • Read the Agent Index (/agent)
  • Discuss a search system (/engage)

Structured data

TechArticle · BreadcrumbList

Change log

Claims

  • Fact · High confidence · checked 2026-07-14

    OpenAI documents three separate bots, each independently controllable in robots.txt: OAI-SearchBot for ChatGPT search inclusion, GPTBot for model training, and ChatGPT-User for user-initiated fetches.

  • Fact · High confidence · checked 2026-07-14

    Google-Extended is a robots.txt token controlling whether content is used to train future Gemini models and for grounding; Google states it does not affect inclusion in Google Search and is not a ranking signal.

  • Fact · High confidence · checked 2026-07-14

    llms.txt is a September 2024 proposal by Jeremy Howard (Answer.AI) for an LLM-friendly markdown index; no major AI platform has confirmed using it, and Google staff have said no AI system currently consumes it.

  • Forecast · Low confidence · checked 2026-07-14

    Agent-mediated visits and actions will remain a small share of most businesses' traffic over the next two years, but will be disproportionately high-intent where they occur.

Sources

Structured-data preview

{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "Organization",
      "@id": "https://www.digitaltraction.co/#organization",
      "name": "Digital Traction",
      "legalName": "Digital Traction LLC",
      "url": "https://www.digitaltraction.co",
      "logo": "https://www.digitaltraction.co/brand/digital-traction-mark.svg",
      "description": "Digital Traction is an independent search-systems practice. It studies and builds how businesses are understood, surfaced, advertised and measured across paid search, organic discovery and emerging AI interfaces.",
      "founder": {
        "@id": "https://www.digitaltraction.co/about#person-sal-ferrara-loris"
      }
    },
    {
      "@type": "Person",
      "@id": "https://www.digitaltraction.co/about#person-sal-ferrara-loris",
      "name": "Sal Ferrara-Loris",
      "jobTitle": "Operator, Digital Traction",
      "description": "Sal Ferrara-Loris operates Digital Traction, an independent search-systems practice covering paid search, relevance architecture, measurement and answer-engine visibility.",
      "knowsAbout": [
        "Paid search",
        "Intent taxonomy",
        "Conversion measurement",
        "Offline conversion imports",
        "Answer-engine visibility",
        "Agent-ready websites"
      ],
      "worksFor": {
        "@id": "https://www.digitaltraction.co/#organization"
      }
    },
    {
      "@type": "BreadcrumbList",
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "Digital Traction",
          "item": "https://www.digitaltraction.co/"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "Systems",
          "item": "https://www.digitaltraction.co/systems"
        },
        {
          "@type": "ListItem",
          "position": 3,
          "name": "Agent-ready web",
          "item": "https://www.digitaltraction.co/systems/agent-ready-web"
        }
      ]
    },
    {
      "@type": "TechArticle",
      "@id": "https://www.digitaltraction.co/systems/agent-ready-web#webpage",
      "mainEntityOfPage": "https://www.digitaltraction.co/systems/agent-ready-web",
      "headline": "Agent-ready web: websites usable by people and software agents",
      "description": "What makes a website usable by both people and agents: semantic HTML, accessibility tree, stable labeled interfaces, structured entities, bounded actions, crawler controls, and the honest limits of llms.txt.",
      "datePublished": "2026-07-14",
      "dateModified": "2026-07-14",
      "author": {
        "@id": "https://www.digitaltraction.co/about#person-sal-ferrara-loris"
      },
      "publisher": {
        "@id": "https://www.digitaltraction.co/#organization"
      }
    }
  ]
}