Does A/B Testing Affect How AI Engines Cite Your Site?

In the most common setup, an A/B test does not affect how AI engines cite your site, because the AI never sees the test: almost every CRO tool swaps content with JavaScript in the browser, and none of the major AI crawlers execute JavaScript. That changes when the test runs server-side, and then the usual rules against cloaking apply. This guide, part of the generative engine optimization guide, covers what server-log evidence shows about AI crawlers, the practical difference between testing in the browser and testing on the server, the four technical precautions Google documentation requires of any test, and the problem almost nobody handles honestly: the AI-sourced traffic segment almost never has the volume to reach sample size, so it cannot be the metric that decides your test.

What AI crawlers can actually read

The underlying question has a technical answer, not a philosophical one: for an AI to cite your page, some crawler has to have read that content. And what those crawlers read is far more limited than most people assume.

A study by Vercel in partnership with MERJ, published in December 2024 from real server logs across the Vercel network, measured the behavior of the main AI crawlers. The central finding: none of the major AI crawlers render JavaScript. OpenAI’s GPTBot fetched JavaScript files on 11.50% of its requests, and the Anthropic crawler on 23.84%, but neither executes those files (Vercel, The rise of the AI crawler). AppleBot is the relevant exception: it renders JavaScript through a browser-based crawler, similar to Googlebot.

The volume helps size the topic: over the analyzed period, GPTBot made 569 million requests across the Vercel network and the Anthropic crawler 370 million, against 4.5 billion from Googlebot. AI crawlers are already meaningful traffic, but still a fraction of Googlebot, and they see less.

Client-side vs server-side: the distinction that settles everything

The entire practical answer to this question fits inside that distinction.

In a client-side test, the server hands the same HTML to everyone and a script rewrites the page in the visitor’s browser. It is the default format of practically every snippet-based CRO tool, Donnu included. Since the AI crawler does not execute that script, it reads exactly the original HTML: the control. To it, the variation does not exist.

In a server-side test, the server decides which version to build before responding to the request. The crawler receives real HTML, and that HTML may be the variation. This is where a test can start to change what an AI reads and eventually cites.

That distinction is the same one that separates the two models in the client-side vs server-side A/B testing guide, read through a different lens: not latency or flicker, but who can read what.

Scenario	What the AI crawler sees	Cloaking risk	What to do
Snippet-based client-side test (the CRO default)	Always the control	None, the HTML is the same for everyone	Nothing beyond the usual; the variation simply does not exist for it
Server-side test with a random draw	Control or variation, depending on the draw	None, the draw does not look at who asked	Keep the draw independent of user agent and IP
Server-side test that forces the bot onto the control	Always the control, by a bot-based decision	High, this is the definition of cloaking	Do not do it; remove the user-agent exception
Split URL with indexable variation URLs	Whichever URL is drawn	Low, if configured properly	rel="canonical" to the original URL and a 302 redirect

The four technical precautions Google documentation requires

When the test touches what is served (server-side or split URL), Google’s official documentation on website testing is short and direct (Google Search Central, A/B testing best practices for Search):

1. Never cloak. The rule is “do not show one set of URLs to Googlebot and another to people”. It applies whether the decision is made by server logic, by robots.txt or by any other means. A test with a genuine random draw does not violate it, because it does not look at who is requesting the page.
2. Use rel="canonical" on variation URLs. If the test uses different URLs, each alternative URL should point by canonical to the original URL. Google recommends this over noindex, because it reflects the intent better: you do not want the page to stop being indexed, you want the versions to be understood as variations grouped under the original.
3. Use a 302 redirect, never a 301. A 302 (temporary) says the original URL should stay in the index. A 301 (permanent) signals the original was replaced, which is exactly the opposite of what a temporary test means.
4. End the test as soon as it is over. Once the experiment finishes, update the site with the chosen variation and remove the test elements. A test left running indefinitely may be interpreted as an attempt to deceive search engines.

There is a fifth point the same documentation records and almost nobody remembers: Googlebot generally does not support cookies. A cookie-controlled test tends to show it only the version a person without cookies would see, normally the control. It is not a problem, it is a consequence worth knowing before trying to explain why the variation “never showed up in the index”.

The consequence almost nobody draws: the test does not expose the improvement

Here is the counterintuitive part. If you run a client-side test, the AI keeps reading the control for the entire experiment. That is comfortable in the short term (nothing you test can “confuse” a generative engine), but it has a practical cost: the winning variation only starts to exist for AI systems once it is promoted to genuinely served content, in the page HTML.

That reorders the work in a useful way. The A/B test proves the change converts better with people. The permanent implementation in HTML is the step that also exposes it to crawlers and generative engines. Treating the two as a single event, “I will test it and see whether AI cites me more”, conflates two processes that happen on different timelines through different mechanisms.

The sample problem: AI traffic is too small to decide

Suppose you want to go further and test whether a content-structure change (the subject of the generative engine optimization guide) improves conversion for people arriving from an assistant. The intent is good, but the arithmetic almost never works.

A site with 12,000 visits a week, of which 320 come from AI assistants, with a baseline conversion rate of 2.6%, wanting to detect a 20% relative improvement (taking the rate to about 3.1%), at 95% confidence and 80% power, needs:

Current conversion rate%Minimum detectable effectrelative (%)absolute (pp)Confidence909599%Power8090%Visitors per week (total)Testtwo-sidedone-sided

-Visitors per variation

-Total (2 variations)

-Estimated duration

Two-proportion normal approximation, 2 variations (50/50). Tweak the inputs and watch it update live.

Set the calculator above to a 2.6 baseline rate, a 20 minimum detectable effect (relative) and 12,000 visitors per week, and the result is 16,128 visits per variation (32,256 in total), which takes about 19 days running across the whole site. Now repeat the arithmetic with the same 16,128 per variation, but fed only by the 320 weekly visits coming from AI: that would be about 706 days, nearly two years, for a single test. In that window the behavior of the assistants themselves would have changed several times over.

You can invert the question and ask what the AI segment can detect within a wait you are willing to accept. At 320 visits a week over 8 weeks, you accumulate 1,280 visits per variation. At that size, on a 2.6% baseline, the minimum detectable effect is roughly 67.8% relative (a target rate of about 4.36%). In other words, only an enormous difference would show up, and enormous conversion differences rarely come out of a content-structure change.

Current conversion rate%Visitors per week (total)Duration (weeks)Variations234Confidence909599%Power8090%Testtwo-sidedone-sided

-Smallest detectable effect (relative)

-In points (absolute)

-Target rate to beat

Sample per variation: -. Two-proportion normal approximation, even split across variations. Tweak the inputs and see the smallest effect your traffic can prove.

Set the calculator above to a 2.6 baseline rate, 320 visitors per week, 8 weeks and 2 variations to reproduce that number. The practical reading: the AI segment is a secondary read, never the decision metric. Run the test across the whole site, decide on it, and look at the AI segment as directional context, knowing it does not have the statistical power to support a decision on its own. It is the same discipline described in the complete conversion rate optimization guide: when volume cannot fund the precision, change what you promise to measure, not the rigor.

How to actually measure AI citation (outside the A/B test)

If citation cannot be the primary metric of an A/B test, it still needs to be tracked somehow. Four signals, none sufficient on its own:

Signal	What it shows	What it does NOT prove
AI crawler hits in server logs	That your page is being read by those systems	That it was cited in any answer
Referral traffic from assistants in analytics	That someone clicked through from a generated answer	The total citation volume, since much of it produces no click
Periodic manual checks on target queries	Whether your page appears cited today, on that specific query	Stability, because the answer changes between runs
Trend in branded search	Whether the exposure in answers is producing recall	Direct causality with any specific change

The honest point: none of those four has the granularity or the volume an A/B test demands. They make up a monitoring dashboard, not an experiment.

The most common mistakes at this intersection

Mistake	Warning sign	Fix
Hiding the test from the bot by user agent	“I configured Googlebot to always see the control”	That is cloaking by Google’s definition; keep the draw random and independent of who asked
Using a 301 in a split URL test	The test redirect is permanent	Switch to 302; a 301 says the original URL was replaced
Expecting a client-side test to change citation	“I tested for three months and the AI still does not cite the new version”	It never read the new version; only served HTML counts
Deciding the test on the AI segment	“In AI traffic, variation B won”	Check the sample on that slice; it almost always lacks statistical power
Leaving the test running indefinitely	The experiment has been live for months with no decision	Google’s own documentation asks that tests be ended and removed when finished
Treating citation as a conversion metric	The dashboard mixes citation with revenue	Citation is an exposure signal; conversion is what the A/B test decides

Make this automatic with Donnu

You just saw that the question in the title has a technical answer rather than an opinion: it depends on where the variation is decided, and the snippet-based A/B test, the most common format, is invisible to the crawlers that feed generative systems. What Donnu solves is the other side of the ledger, the side that actually decides: measuring conversion for real people across the whole site, and returning an honest verdict with the confidence interval in front, holding back the winner call until the variation has at least 200 visitors and 7 days on air, instead of letting you declare a winner on a traffic slice far too small to support any conclusion.

Start a 14-day free trial and bring that rigor to your content changes. For the full background, see the generative engine optimization guide.

References

• Vercel and MERJ. The rise of the AI crawler. Server-log analysis published on December 17, 2024. vercel.com/blog/the-rise-of-the-ai-crawler.
• Google Search Central. A/B testing best practices for Search. developers.google.com/search/docs/crawling-indexing/website-testing.
• Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A. GEO: Generative Engine Optimization. ACM SIGKDD 2024. arxiv.org/abs/2311.09735.

Read next:

• Generative engine optimization (GEO): the 2026 guide
• Client-side vs server-side A/B testing: the complete comparison
• How to implement server-side A/B testing (step by step)
• A/B testing statistical significance: the complete guide