Can AI Agents Read Your Website?
Learn about how to improve your website into AI readable. This article includes 9 AI Accessibility Check lists and 5 Real-World Page Audits.
Learn about how to improve your website into AI readable. This article includes 9 AI Accessibility Check lists and 5 Real-World Page Audits.
Learn about how to improve your website into AI readable. This article includes 9 AI Accessibility Check lists and 5 Real-World Page Audits.
Learn about how to improve your website into AI readable. This article includes 9 AI Accessibility Check lists and 5 Real-World Page Audits.
Learn about how to improve your website into AI readable. This article includes 9 AI Accessibility Check lists and 5 Real-World Page Audits.
ChatGPT, Gemini, Perplexity, Google AI Overviews, and Naver AI Briefing all build their answers by finding, reading, and citing web pages. But a page that looks fine to people is not necessarily readable by AI bots. To become a citation candidate, a page has to clear three gates. The bot must be able to reach the page, the HTML the server sends first must actually contain the content, and that content must be text rather than images. This guide explains why each gate fails, how BubbleShare improved AI citations for its own blog, what the nine AI accessibility checks are, and what we found when we audited five pages from five different industries. It is based on BubbleShare's September 2026 webinar, "Can AI Agents Read Your Website?"
Sites that block AI search bots do not get cited in AI answers. AI engines combine information they have already collected with information they find through live web search, and AI bots are what fetch that information. If the bots are blocked, the AI cannot use your page no matter how useful or well-structured it is. Data published by Cloudflare shows the same pattern: citation propensity, the number of AI answer citations per Google search impression, dropped to zero for sites that blocked AI bots.
Yet when you open the sites that block AI bots, most of them never meant to block AI. To see why, you first need to separate the two kinds of AI bots.
| Training bots | Search bots | |
|---|---|---|
| Role | Collect data to train the next AI model | Fetch pages in real time to answer a user's question |
| If blocked | Your content is excluded from training data, but you can still be cited in AI answers | Your brand and content are not referenced in AI answers |
| Should you block it? | A reasonable choice if you worry about inaccurate training on brand information or server load | No reason to block |
| Examples | GPTBot, Google-Extended, ClaudeBot | OAI-SearchBot, PerplexityBot, Claude-SearchBot |
The real problem is robots.txt rules written before AI bots existed. robots.txt is the file that tells bots what they may and may not fetch from a site. Years ago, many companies allowed only Google's crawler and blocked everything else to stop unwanted scraping by price comparison sites and competitor monitoring bots.
User-agent: Googlebot
Allow: /
User-agent: *
Disallow: /
Left as is, this rule blocks both training bots and search bots. Because AI bots did not exist when it was written, nobody updated it, and many sites still block the very search bots that lead to AI citations. The rule may also be enforced by internal security policy, so it is worth checking directly at least once.
The cause is usually rendering. Rendering is the process in which a browser takes the HTML document sent by the server and draws it as the screen people see. The HTML holds the page title, body text, prices, reviews, and other information. People see the finished screen, but AI bots read the HTML document the server sends first.
Some sites send that HTML almost empty and let the browser fill in titles, prices, and reviews afterward. This approach, commonly called client-side rendering (CSR), became popular over the past several years because it makes page transitions fast and app-like. It causes no problems for people, but most AI bots never run the fill-in step. They reach the page and leave with an empty shell.
| Who reads the page | What they see | Result |
|---|---|---|
| People (browser) | The finished screen with all content filled in | Product names, prices, and reviews are all visible |
| Google's crawler | A screen rendered on its behalf | The page appears normally in Google Search |
| Most AI search bots | The HTML the server sent first | The content is empty, so there is nothing to cite |
Because Google runs the rendering step itself, a page can perform well in Google Search while being absent from ChatGPT and other AI engines.
In BubbleShare's internal project data, citation frequency between text-based and image-based product detail pages differed by as much as roughly 100 times. AI bots can currently take only text. For years, product detail pages were commonly built as one long image because it looked clean on mobile. People can read the information inside the image, but to an AI bot that page is effectively empty. The same product information is cited far more often when the page includes it as text.
BubbleShare set a 2026 goal of being recognized by prospects as "the team that spreads content the furthest, powered by its own solution." To turn that goal into GEO content and verify the citation impact, we worked through three steps.
Optimization starts with deciding which message you want AI engines to associate with you.
We first defined the keyword areas we wanted to appear in (GEO and SEO, and localization) and wrote a representative statement for each area describing how we want AI to explain BubbleShare. We then mapped the questions that statement would answer and the people asking them. Knowing who asks is what tells you which topics and formats to create.
| Design element | Example for the GEO/SEO area |
|---|---|
| Representative statement | A GEO/AIO solution that measures AI search visibility and connects it to execution |
| Expected question | How do I get my brand to appear in AI search engines? |
| Who asks | Marketers, PR managers, and brand managers who want their content cited |
| Citation evidence: measurability | Analysis of 100,000 AI answers; share-of-mention and citation metrics |
| Citation evidence: execution | Technical audits of 170 pages; pre-publication scoring |
| Citation evidence: proof | 59.1% citation rate for our own pages; new content consistently scoring 90+ on key metrics |
For an AI engine to give a particular answer, the content must contain concrete facts the AI can use as evidence. That is why we paired the representative statement with figures supporting measurability, execution, and proof rather than repeating the statement itself.
In our monitoring solution, VIVI, we registered six related topics and added the related questions and keywords VIVI recommends for each, for a total of 76 questions. We then measured how often and how prominently we appeared for those questions in Naver AI Briefing, Google AI Overviews, Gemini, ChatGPT, and Perplexity.
Two problems surfaced.
First, citations were happening across a wider range than we had targeted. The estimated monthly search volume of the 12 keywords where we were actually cited differed by nearly 2x from the combined search volume of the keywords in our target questions. We went beyond our original topics and keywords and registered additional questions and topics.
Second, our uncited pages had no way to be fixed. At the time, the BubbleShare blog was published on Notion, and the only thing we could edit was the page description. We could not structure URLs so AI engines would recognize them, could not tell AI bots what an image showed through alt text, and could not control page titles or structure in the metadata. Notion let non-developers write and publish quickly, but it severely limited SEO, technical improvements, and any work to make pages easier for AI bots to read.
| Before (Notion-based) | After | |
|---|---|---|
| Editable elements | Page description only | Every element: URLs, image alt text, metadata, heading structure |
| Content review | No GEO check before publishing | GEO scoring with VIVI Content Assistant before and after publishing |
| Performance tracking | No way to track citations | Pages registered in VIVI monitoring to track citation performance |
We first moved the blog to a platform where we could edit every element described above. We then turned publishing into a loop: check the GEO score with VIVI Content Assistant before publishing, check it again after publishing, and improve what is needed. Newly published content scored 94 on citation potential, and those pages are registered in VIVI monitoring alongside the questions from Step 2 so we can keep tracking their citations. With that foundation in place, we produced and published new content for the keywords and topics we had not covered before.
AI accessibility covers two areas. On the technical side, check whether the page responds without errors, whether it blocks AI bots, whether a noindex tag is refusing indexing, and whether the HTML sent by the server contains the body content. On the content SEO side, check the title tag, metadata, canonical URL, structured data (schema), heading tags, and author and publication details. BubbleShare groups these into nine checks across three stages: reach, understanding, and trust.
| Stage | Check | What to look for | What happens if it fails |
|---|---|---|---|
| Reach | AI crawler access | Can AI bots reach the page? | Bots cannot enter the page |
| Reach | Structured data | Is the page's information typed and structured with schema? | Content is hard or slow to parse, and information gets dropped |
| Reach | Metadata | Does the page convey its topic and summary to AI? | AI misreads the context and generates answers with errors |
| Understanding | Content structure | Is the heading order defined with heading tags? | The body is read as one block, so information cannot be taken section by section |
| Understanding | Answer directness | Are there sentences that answer a question directly, conclusion-first sections, and summary blocks? | Citable passages are missed |
| Understanding | Content originality | Does the page contain unique data and cases rather than copied or translated content? | Only the original, earlier-published source gets cited |
| Trust | Author and publication details | Are the author, publication date, and last-updated date shown? | The content is judged less trustworthy |
| Trust | Freshness | Is an update history shown? | The content is treated as old and unmaintained |
| Trust | Fact-checkability | Are original sources, in-house research, and sample sizes stated for figures? | Citations decline because the evidence looks thin |
The reach stage checks whether AI bots can get to the page and understand it quickly. When all three checks pass, the AI can enter the page and immediately tell what it is about.
The understanding stage checks whether the page can serve as material for answering a user's question. Splitting content into sections is not enough on its own. You need to mark them with heading tags (this is the main title, H1; this is a section heading, H2; this is a subheading, H3) so AI bots can organize and take information section by section. Placing the conclusion first under a question-style heading and the details after it helps AI find the sentence that answers the question. Originality matters too. Content written by referencing other content hands the citation to the original, so you need to mix in your own data and cases that exist nowhere else, such as how many pages you analyzed or what citation results you achieved.
The trust stage checks whether the AI has grounds to treat the content as citable. This is why major publishers and platforms around the world show the author, publication date, and update date by default. Content with a visible update history is treated as actively maintained. When a page states figures without sources, AI engines cannot tell whether the numbers are correct and hesitate to cite them.
From the pages submitted during webinar registration, we picked five that differed in industry and page type and audited them with VIVI Content Assistant. All five pages were accessible to AI bots. The differences appeared in the next stages: structured data, metadata, content structure, and text availability. Brand names are anonymized.
The page had strong content, including advertising performance awards, but lacked the signals AI needs to understand it accurately.
The fact that most of the body was text was a real advantage for AI readability.
Instead of the homepage, we audited the product listing page for one product category.
Instead of the homepage, we audited a photo review event page.
Shoppers really do ask AI questions like "Are there any coupons or discounts on this brand's fall collection?" When the benefits sit inside an image, they cannot become part of that answer.
We also ran VIVI Content Assistant's technical fix generation feature on this page. Based on the audit, it suggests ready-to-paste FAQ schema (JSON-LD), a revised meta title and description, and alt text for each image. It also generates instructions you can paste into another AI tool to revise the page according to the recommendations.
Most checks scored well, but the page had problems typical of product detail pages.
The deep analysis available in the paid version went a step further. It pointed out that although the full ingredient list was present, it was hard to match claims like "contains rose oil" against the actual extract names in the ingredient list, and that figures such as clinical results and wear time had no research source. These issues tie directly to fact-checkability in the trust stage. This information is usually already inside the product page images, so simply moving it into text raises the chance of being cited in answers to customer questions.
| Case | Page type | Main issues | Priority fixes |
|---|---|---|---|
| 1 | Agency service page | No schema, no FAQ schema, metadata copied from homepage, no H1 | Add service and FAQ schema; write page-specific metadata |
| 2 | Appliance product listing (PLP) | Meaningless URL, no structured data, metadata copied from homepage, no H1 | Category-based URL, product schema, category-specific metadata |
| 3 | Fashion review event | No H1, benefits and terms inside an image | Set the event name as H1; convert benefits to text |
| 4 | Brokerage ETF guide | No schema, no FAQ schema, no OG image, too many H1s | Add FAQ schema; reduce to a single H1 |
| 5 | Beauty product detail page (PDP) | Shade, ingredient, and clinical information inside images; no freshness signal; no sources | Convert key image content to text; cite sources for figures |
H1 errors appeared on four of the five pages, and homepage-copied metadata and on-screen-only FAQs each appeared on two. Every one of these can be fixed by changing settings on existing pages, without creating new content.
1. Check whether robots.txt allows AI crawlers. Type yourdomain.com/robots.txt into your browser to see which bots are allowed and which are disallowed. A fix is needed if any of the following apply:
Correcting these settings alone is a starting point for improving AI citations.
2. Move the key messages inside images into text. If replacing a long image all at once is not practical, start small: convert only the messages you must communicate into copy text and keep the rest as images. Adding a short text summary directly below each image also works well.
3. Show the author, publication date, and last-updated date. Add them both in the visible body of the page and in the page's markup (structured data). Markup is invisible on screen but tells AI bots "this is the author" and "this is the publication date."
Q. If I block AI training bots, will my site disappear from AI answers?
A. No. Blocking training bots only removes your content from AI model training data. Your chances of being cited in AI answers stay the same. Citations stop when you block search bots, and sites that blocked them showed a citation propensity of zero.
Q. Why does my page appear in Google Search but not in ChatGPT answers?
A. The likely cause is a rendering method in which the server sends empty HTML and the browser fills in the content later. Google runs that step itself, but most AI bots do not, so they take an empty page.
Q. How much do image-only product detail pages hurt AI citations?
A. In BubbleShare's internal projects, citation frequency between text-based and image-based product detail pages differed by as much as roughly 100 times. AI bots currently take only text.
Q. How many H1 tags should a page have?
A. One. Of the five pages we audited, three had no H1 and one had far too many. Without an H1, the page's topic is unclear; with too many, AI confuses the topic.
Q. Where should I start with AI accessibility?
A. The three fastest fixes are checking whether robots.txt allows AI crawlers, moving key messages inside images into text, and showing the author, publication date, and update date both on screen and in markup.
Whether a page gets cited in AI answers depends first on whether AI can read it, before content quality even comes into play. AI search bots must be able to reach it, the HTML the server sends must contain the content, and key information must exist as text. With the nine checks across reach, understanding, and trust in place on top of that, a page becomes a citation candidate. In our audits of five pages across five industries, every page was accessible to AI bots, yet the same problems kept appearing: H1 tags, metadata, FAQ schema, and image-only content, all fixable by changing settings on existing pages. BubbleShare itself moved off a blog platform it could not edit, built a scoring loop before and after publishing, and saw new content score 94 and its own pages reach a 59.1% citation rate.
VIVI is a GEO solution that measures AI search visibility and connects it to execution. Start by seeing how AI reads your pages.