Can AI Agents Read Your Website?

Learn about how to improve your website into AI readable. This article includes 9 AI Accessibility Check lists and 5 Real-World Page Audits.

20 min read
by Sooah Lee

ChatGPT, Gemini, Perplexity, Google AI Overviews, and Naver AI Briefing all build their answers by finding, reading, and citing web pages. But a page that looks fine to people is not necessarily readable by AI bots. To become a citation candidate, a page has to clear three gates. The bot must be able to reach the page, the HTML the server sends first must actually contain the content, and that content must be text rather than images. This guide explains why each gate fails, how BubbleShare improved AI citations for its own blog, what the nine AI accessibility checks are, and what we found when we audited five pages from five different industries. It is based on BubbleShare's September 2026 webinar, "Can AI Agents Read Your Website?"

What happens to AI citations when you block AI search bots?

Sites that block AI search bots do not get cited in AI answers. AI engines combine information they have already collected with information they find through live web search, and AI bots are what fetch that information. If the bots are blocked, the AI cannot use your page no matter how useful or well-structured it is. Data published by Cloudflare shows the same pattern: citation propensity, the number of AI answer citations per Google search impression, dropped to zero for sites that blocked AI bots.

Yet when you open the sites that block AI bots, most of them never meant to block AI. To see why, you first need to separate the two kinds of AI bots.

Training botsSearch bots
RoleCollect data to train the next AI modelFetch pages in real time to answer a user's question
If blockedYour content is excluded from training data, but you can still be cited in AI answersYour brand and content are not referenced in AI answers
Should you block it?A reasonable choice if you worry about inaccurate training on brand information or server loadNo reason to block
ExamplesGPTBot, Google-Extended, ClaudeBotOAI-SearchBot, PerplexityBot, Claude-SearchBot

The real problem is robots.txt rules written before AI bots existed. robots.txt is the file that tells bots what they may and may not fetch from a site. Years ago, many companies allowed only Google's crawler and blocked everything else to stop unwanted scraping by price comparison sites and competitor monitoring bots.

User-agent: Googlebot
Allow: /

User-agent: *
Disallow: /

Left as is, this rule blocks both training bots and search bots. Because AI bots did not exist when it was written, nobody updated it, and many sites still block the very search bots that lead to AI citations. The rule may also be enforced by internal security policy, so it is worth checking directly at least once.

Why does a page rank on Google but never appear in ChatGPT answers?

The cause is usually rendering. Rendering is the process in which a browser takes the HTML document sent by the server and draws it as the screen people see. The HTML holds the page title, body text, prices, reviews, and other information. People see the finished screen, but AI bots read the HTML document the server sends first.

Some sites send that HTML almost empty and let the browser fill in titles, prices, and reviews afterward. This approach, commonly called client-side rendering (CSR), became popular over the past several years because it makes page transitions fast and app-like. It causes no problems for people, but most AI bots never run the fill-in step. They reach the page and leave with an empty shell.

Who reads the pageWhat they seeResult
People (browser)The finished screen with all content filled inProduct names, prices, and reviews are all visible
Google's crawlerA screen rendered on its behalfThe page appears normally in Google Search
Most AI search botsThe HTML the server sent firstThe content is empty, so there is nothing to cite

Because Google runs the rendering step itself, a page can perform well in Google Search while being absent from ChatGPT and other AI engines.

How much do image-only product pages hurt AI citations?

In BubbleShare's internal project data, citation frequency between text-based and image-based product detail pages differed by as much as roughly 100 times. AI bots can currently take only text. For years, product detail pages were commonly built as one long image because it looked clean on mobile. People can read the information inside the image, but to an AI bot that page is effectively empty. The same product information is cited far more often when the page includes it as text.

How did BubbleShare improve AI citations for its own blog?

BubbleShare set a 2026 goal of being recognized by prospects as "the team that spreads content the furthest, powered by its own solution." To turn that goal into GEO content and verify the citation impact, we worked through three steps.

Optimization starts with deciding which message you want AI engines to associate with you.

Step 1: We defined how we want AI to describe us

We first defined the keyword areas we wanted to appear in (GEO and SEO, and localization) and wrote a representative statement for each area describing how we want AI to explain BubbleShare. We then mapped the questions that statement would answer and the people asking them. Knowing who asks is what tells you which topics and formats to create.

Design elementExample for the GEO/SEO area
Representative statementA GEO/AIO solution that measures AI search visibility and connects it to execution
Expected questionHow do I get my brand to appear in AI search engines?
Who asksMarketers, PR managers, and brand managers who want their content cited
Citation evidence: measurabilityAnalysis of 100,000 AI answers; share-of-mention and citation metrics
Citation evidence: executionTechnical audits of 170 pages; pre-publication scoring
Citation evidence: proof59.1% citation rate for our own pages; new content consistently scoring 90+ on key metrics

For an AI engine to give a particular answer, the content must contain concrete facts the AI can use as evidence. That is why we paired the representative statement with figures supporting measurability, execution, and proof rather than repeating the statement itself.

Step 2: We tracked 76 questions across five AI engines

In our monitoring solution, VIVI, we registered six related topics and added the related questions and keywords VIVI recommends for each, for a total of 76 questions. We then measured how often and how prominently we appeared for those questions in Naver AI Briefing, Google AI Overviews, Gemini, ChatGPT, and Perplexity.

Two problems surfaced.

First, citations were happening across a wider range than we had targeted. The estimated monthly search volume of the 12 keywords where we were actually cited differed by nearly 2x from the combined search volume of the keywords in our target questions. We went beyond our original topics and keywords and registered additional questions and topics.

Second, our uncited pages had no way to be fixed. At the time, the BubbleShare blog was published on Notion, and the only thing we could edit was the page description. We could not structure URLs so AI engines would recognize them, could not tell AI bots what an image showed through alt text, and could not control page titles or structure in the metadata. Notion let non-developers write and publish quickly, but it severely limited SEO, technical improvements, and any work to make pages easier for AI bots to read.

Step 3: We changed both the platform and the publishing process

Before (Notion-based)After
Editable elementsPage description onlyEvery element: URLs, image alt text, metadata, heading structure
Content reviewNo GEO check before publishingGEO scoring with VIVI Content Assistant before and after publishing
Performance trackingNo way to track citationsPages registered in VIVI monitoring to track citation performance

We first moved the blog to a platform where we could edit every element described above. We then turned publishing into a loop: check the GEO score with VIVI Content Assistant before publishing, check it again after publishing, and improve what is needed. Newly published content scored 94 on citation potential, and those pages are registered in VIVI monitoring alongside the questions from Step 2 so we can keep tracking their citations. With that foundation in place, we produced and published new content for the keywords and topics we had not covered before.

What are the 9 AI accessibility checks, and what should you look for?

AI accessibility covers two areas. On the technical side, check whether the page responds without errors, whether it blocks AI bots, whether a noindex tag is refusing indexing, and whether the HTML sent by the server contains the body content. On the content SEO side, check the title tag, metadata, canonical URL, structured data (schema), heading tags, and author and publication details. BubbleShare groups these into nine checks across three stages: reach, understanding, and trust.

StageCheckWhat to look forWhat happens if it fails
ReachAI crawler accessCan AI bots reach the page?Bots cannot enter the page
ReachStructured dataIs the page's information typed and structured with schema?Content is hard or slow to parse, and information gets dropped
ReachMetadataDoes the page convey its topic and summary to AI?AI misreads the context and generates answers with errors
UnderstandingContent structureIs the heading order defined with heading tags?The body is read as one block, so information cannot be taken section by section
UnderstandingAnswer directnessAre there sentences that answer a question directly, conclusion-first sections, and summary blocks?Citable passages are missed
UnderstandingContent originalityDoes the page contain unique data and cases rather than copied or translated content?Only the original, earlier-published source gets cited
TrustAuthor and publication detailsAre the author, publication date, and last-updated date shown?The content is judged less trustworthy
TrustFreshnessIs an update history shown?The content is treated as old and unmaintained
TrustFact-checkabilityAre original sources, in-house research, and sample sizes stated for figures?Citations decline because the evidence looks thin

The reach stage checks whether AI bots can get to the page and understand it quickly. When all three checks pass, the AI can enter the page and immediately tell what it is about.

The understanding stage checks whether the page can serve as material for answering a user's question. Splitting content into sections is not enough on its own. You need to mark them with heading tags (this is the main title, H1; this is a section heading, H2; this is a subheading, H3) so AI bots can organize and take information section by section. Placing the conclusion first under a question-style heading and the details after it helps AI find the sentence that answers the question. Originality matters too. Content written by referencing other content hands the citation to the original, so you need to mix in your own data and cases that exist nowhere else, such as how many pages you analyzed or what citation results you achieved.

The trust stage checks whether the AI has grounds to treat the content as citable. This is why major publishers and platforms around the world show the author, publication date, and update date by default. Content with a visible update history is treated as actively maintained. When a page states figures without sources, AI engines cannot tell whether the numbers are correct and hesitate to cite them.

What did audits of 5 pages across 5 industries reveal?

From the pages submitted during webinar registration, we picked five that differed in industry and page type and audited them with VIVI Content Assistant. All five pages were accessible to AI bots. The differences appeared in the next stages: structured data, metadata, content structure, and text availability. Brand names are anonymized.

Case 1. A performance marketing agency's service page

The page had strong content, including advertising performance awards, but lacked the signals AI needs to understand it accurately.

  • The page type was undefined. It needs a schema type (JSON-LD) that tells AI whether this is a service page or a product page.
  • The FAQ section existed only on screen. It already contained questions brands and agencies often ask AI, such as platform-specific strategies for reaching younger audiences. Adding them as FAQ schema in the page code would let AI read the questions and answers precisely.
  • The meta description was shorter than recommended and identical to the homepage. A title and description specific to this page, highlighting its results and operational strength, would help AI recognize the agency as a proven operator.
  • There was no H1, only H3 tags. Heading tags should be assigned to text that actually appears on the page. The official partner label or the core introductory sentence at the top could become the H1, and English-only text could be paired with a Korean version for the Korean market.

The fact that most of the body was text was a real advantage for AI readability.

Case 2. A kitchen appliance brand's product listing page (PLP)

Instead of the homepage, we audited the product listing page for one product category.

  • The URL gave no indication of the category. A URL structure that reveals the category would help.
  • It was a product page with no structured data.
  • The metadata was identical to the homepage. It needs a title and description matched to the category.
  • No H1 was assigned.
  • Author and update information was missing. Two of the trust-stage checks were empty.

Case 3. A fashion brand's review event page

Instead of the homepage, we audited a photo review event page.

  • There was no H1. The event name at the top of the page should become the H1, with English-only text paired with Korean.
  • The meta description was well written. It mentioned the photo review event and the double rewards for writing a review, so AI could grasp the main idea of the event.
  • The key benefits and terms were inside a single image. AI could not read specific benefits such as "20,000 points for top reviewers twice a month" or the event terms.

Shoppers really do ask AI questions like "Are there any coupons or discounts on this brand's fall collection?" When the benefits sit inside an image, they cannot become part of that answer.

Case 4. A brokerage firm's ETF guide page

  • There was no schema at all. The page type was not specified.
  • The "What is an ETF?" FAQ existed only on screen. FAQ schema should be added to the page code.
  • Metadata scored 89, but there was no Open Graph image. When shared on social media, the link shows only a title and description with no thumbnail, so a large image card (summary_large_image) setting is needed.
  • There were too many H1s. This is the opposite of the earlier cases. On an ETF guide page, menu items such as account opening, banking, and paper trading were all tagged H1, which can confuse AI about the page's topic. A page should have exactly one H1.

We also ran VIVI Content Assistant's technical fix generation feature on this page. Based on the audit, it suggests ready-to-paste FAQ schema (JSON-LD), a revised meta title and description, and alt text for each image. It also generates instructions you can paste into another AI tool to revise the page according to the recommendations.

Case 5. A beauty brand's global store product detail page (PDP) for a lip tint

Most checks scored well, but the page had problems typical of product detail pages.

  • The meta description was too long. It should be shortened to the recommended length.
  • Freshness signals were missing.
  • Color information was inside images. When a user asks AI which shades are available, or when AI tries to show the shades visually, it cannot use this information.
  • The only text was part of the product features at the bottom. Only features such as rose oil, a non-sticky feel, and smooth application were written as text.

The deep analysis available in the paid version went a step further. It pointed out that although the full ingredient list was present, it was hard to match claims like "contains rose oil" against the actual extract names in the ingredient list, and that figures such as clinical results and wear time had no research source. These issues tie directly to fact-checkability in the trust stage. This information is usually already inside the product page images, so simply moving it into text raises the chance of being cited in answers to customer questions.

Which problems repeated across the five cases?

CasePage typeMain issuesPriority fixes
1Agency service pageNo schema, no FAQ schema, metadata copied from homepage, no H1Add service and FAQ schema; write page-specific metadata
2Appliance product listing (PLP)Meaningless URL, no structured data, metadata copied from homepage, no H1Category-based URL, product schema, category-specific metadata
3Fashion review eventNo H1, benefits and terms inside an imageSet the event name as H1; convert benefits to text
4Brokerage ETF guideNo schema, no FAQ schema, no OG image, too many H1sAdd FAQ schema; reduce to a single H1
5Beauty product detail page (PDP)Shade, ingredient, and clinical information inside images; no freshness signal; no sourcesConvert key image content to text; cite sources for figures

H1 errors appeared on four of the five pages, and homepage-copied metadata and on-screen-only FAQs each appeared on two. Every one of these can be fixed by changing settings on existing pages, without creating new content.

What 3 AI accessibility fixes can you apply today?

1. Check whether robots.txt allows AI crawlers. Type yourdomain.com/robots.txt into your browser to see which bots are allowed and which are disallowed. A fix is needed if any of the following apply:

  • An AI bot's name is followed by Disallow.
  • All bots (User-agent: *) are disallowed, with only Google or Naver allowed separately.
  • There is no robots.txt file at all.

Correcting these settings alone is a starting point for improving AI citations.

2. Move the key messages inside images into text. If replacing a long image all at once is not practical, start small: convert only the messages you must communicate into copy text and keep the rest as images. Adding a short text summary directly below each image also works well.

3. Show the author, publication date, and last-updated date. Add them both in the visible body of the page and in the page's markup (structured data). Markup is invisible on screen but tells AI bots "this is the author" and "this is the publication date."

Frequently asked questions about AI accessibility

Q. If I block AI training bots, will my site disappear from AI answers?

A. No. Blocking training bots only removes your content from AI model training data. Your chances of being cited in AI answers stay the same. Citations stop when you block search bots, and sites that blocked them showed a citation propensity of zero.

Q. Why does my page appear in Google Search but not in ChatGPT answers?

A. The likely cause is a rendering method in which the server sends empty HTML and the browser fills in the content later. Google runs that step itself, but most AI bots do not, so they take an empty page.

Q. How much do image-only product detail pages hurt AI citations?

A. In BubbleShare's internal projects, citation frequency between text-based and image-based product detail pages differed by as much as roughly 100 times. AI bots currently take only text.

Q. How many H1 tags should a page have?

A. One. Of the five pages we audited, three had no H1 and one had far too many. Without an H1, the page's topic is unclear; with too many, AI confuses the topic.

Q. Where should I start with AI accessibility?

A. The three fastest fixes are checking whether robots.txt allows AI crawlers, moving key messages inside images into text, and showing the author, publication date, and update date both on screen and in markup.

Conclusion: AI citations start with a page AI can read

Whether a page gets cited in AI answers depends first on whether AI can read it, before content quality even comes into play. AI search bots must be able to reach it, the HTML the server sends must contain the content, and key information must exist as text. With the nine checks across reach, understanding, and trust in place on top of that, a page becomes a citation candidate. In our audits of five pages across five industries, every page was accessible to AI bots, yet the same problems kept appearing: H1 tags, metadata, FAQ schema, and image-only content, all fixable by changing settings on existing pages. BubbleShare itself moved off a blog platform it could not edit, built a scoring loop before and after publishing, and saw new content score 94 and its own pages reach a 59.1% citation rate.

VIVI is a GEO solution that measures AI search visibility and connects it to execution. Start by seeing how AI reads your pages.

  • VIVI Content Assistant (free browser extension): On any web page, one click shows scores and improvement guidance for all nine AI accessibility checks. You get 30 free credits every month, enough to audit 30 pages, and the technical fix generation feature, which suggests schema code, meta copy, and image alt text, is also available in the free version (it uses more credits). We recommend scanning pages first with a 1-credit analysis, then generating technical fixes only for your most important pages.
  • VIVI Insight: See how often your brand is mentioned in Naver AI Briefing, Google AI Overviews, Gemini, ChatGPT, and Perplexity, and how often the pages you improved are cited. Beyond website pages, you can register Naver blog posts and influencer campaign content to track every week how often they are cited in AI answers to the questions you monitor. The VIVI Starter plan comes with a 7-day free trial.