Where do I install the A/B testing script on a Shopify store?

There are two usual paths. The first is editing the theme and placing the script in theme.liquid, as high as possible inside the head, so it decides the variation before the first paint. The second is using an app embed from the Shopify App Store, when the tool offers one. The theme path gives more control over the exact position of the script, which is what prevents the flicker of the original version appearing before the variation.

How much traffic does a Shopify store need for a test to reach significance?

It depends on the current conversion rate and the minimum gain you want to detect. One reference calculated with the math used across this blog: a store converting 2.2% of sessions into orders that wants to detect a 15% relative improvement needs roughly 33,284 sessions per variation. Aiming at 20% relative, the requirement drops to roughly 19,147 per variation. Stores below that volume should test higher in the funnel, where the baseline rate is larger, such as add to cart or checkout entry.

Why do the visitor numbers in my testing tool not match the Shopify dashboard?

Because they count different things. The Shopify dashboard counts sessions using its own window and bot filtering rules; the testing tool counts unique visitors exposed to a variation, with its own identification rule and its own window. A divergence of a few percentage points is normal and expected. What is not normal is the split between variations drifting from what you configured, which is a sample ratio mismatch and needs investigation before any result is read.

Can I test pricing on Shopify with a client-side tool?

Technically you can swap the displayed price with JavaScript, and it is a bad idea: the displayed price would diverge from the real price charged in the cart and checkout, which come from the Shopify backend. Beyond the consumer protection exposure, the test would measure the reaction to a price the store will not charge. Price testing on Shopify belongs in an app that changes the real catalogue price, or in an automatic discount applied by rule, never in a visual override.

Does a Shopify test need to run in whole weeks?

Yes, and it is worth planning for from the start. Buying behaviour on a Monday is not the behaviour on a Saturday, so a test that starts on a Monday and closes on a Thursday captures a biased slice of the week. Run in multiples of seven days even when the calculated sample fills earlier, and if a shopping holiday or a large paid traffic campaign lands inside the window, record the exact dates and read the result with and without that period.

CRO

A/B Testing on Shopify: The Complete Guide

Shopify a/b testing guide: where the script goes, what you can test outside checkout, how to size with real traffic and the theme traps that break tests.

by Equipe DonnuAugust 9, 202612 min readUpdated on August 9, 2026

Abstract flat illustration of a striped storefront awning above a browser panel split down the middle into two mirrored halves

📚 This article is part of the guide Ecommerce Checkout Optimization: The A/B Testing Playbook.

A/B testing on a Shopify store works very well for everything that happens before checkout, and barely works inside checkout with generic tools, because Shopify closed the checkout to third party scripts and replaced them with extensions it validates itself. This article is part of the complete checkout optimization playbook and covers the process end to end: where to install, what to test on each surface of the store, how to size with the traffic you actually have, and the theme and cart traps that ruin Shopify tests.

If your question is which tool to pick, the dedicated article on A/B testing tools for Shopify compares the categories neutrally. Here the focus is execution.

The store map: where a test reaches and where it does not

The checkout boundary is the single most important fact about A/B testing on Shopify. Everything the theme renders accepts a snippet normally; the checkout accepts only extensions built and validated for it. Script tags on the order status and thank you pages were sunset on 28 August 2025 for Plus stores and, per Shopify documentation, are sunset for non-Plus stores on 26 August 2026.

That is less limiting than it first appears. The leading causes of abandonment, according to the Baymard Institute, are about price transparency, and most of them can be addressed before checkout: show the estimated shipping cost in the cart, make the free shipping threshold explicit, display the full total before the shopper enters the payment flow. The article on cart abandonment A/B testing covers those levers in detail.

Step 1: choose the metric your volume can actually support

Before installing anything, define the number that will decide the test, and define it inside the volume reality of your store. That choice determines how long the test takes more than any other decision.

Metric	Typical baseline	Volume required	When to use it
Session to order	1% to 3%	Very high	Stores with tens of thousands of sessions per week
Session to add to cart	5% to 12%	High	The most common middle ground; still ties to the order, but closes much faster
Cart to checkout entry	40% to 70%	Medium	Cart page tests
Cart to order	25% to 40%	Medium	Cost transparency tests in the cart

The practical rule: the lower the baseline rate, the more traffic the test demands. A store with 10,000 sessions a week cannot close a test decided on session to order in a reasonable window, and can close one decided on add to cart. Choosing the metric is choosing the timeline.

Step 2: install the script in the right place in the theme

The install point decides whether the test works or produces that flicker of the original version before the variation appears, which is both ugly and contaminating (a share of visitors see both versions).

In theme.liquid, as high as possible inside the head. The script has to pick the variation before the first paint. Placing it ahead of stylesheets and other heavy scripts is what prevents the flicker.
Synchronously, or with an anti-flicker snippet. An async script in the footer practically guarantees the flicker. If your tool offers an anti-flicker block, use it, and configure a short timeout so a network failure never leaves the page hidden.
Test on a duplicated theme first. Shopify lets you duplicate the theme and preview it through a dedicated URL. Validate there, confirm the variation renders and the conversion event fires, and only then publish to the live theme.
Without breaking Online Store 2.0 sections. Modern themes assemble the page from sections the merchant can reorder in the editor. A variation that depends on a fixed element position breaks the moment someone reorders sections mid test.

Step 3: what to test on each surface

Surface	Highest leverage tests	Shopify specific caution
Product page	Buy block above the fold, photo order, reviews near the price, button copy	The variant selector is what breaks most often; test every colour and size combination
Collection	Items per row, visible filters, default sort, bestseller badges	Pagination and infinite scroll change the denominator; fix the rule before running
Home	Main hero, block order, category highlights	The most edited page in the store; freeze marketing edits during the test
Cart (drawer or page)	Estimated shipping, free shipping progress bar, complementary upsell	The drawer is updated by AJAX; the variation has to be reapplied on every DOM change
Capture popup	Trigger moment, offer, number of fields	App popups compete with your test; disable the others during the experiment
Checkout	Out of reach for a snippet	Checkout UI Extension only

The cart item deserves emphasis because it is the number one cause of broken Shopify tests. When a shopper adds a product, the drawer is redrawn dynamically and the JavaScript injected variation disappears, unless the script observes DOM changes and reapplies itself. The symptom is always the same: the test “works” in staging (where nobody adds a product) and produces a strangely small effect in production.

Step 4: size before switching on

Adjust the current conversion rate of your chosen metric, the minimum gain that would justify shipping, and the real weekly session volume:

Sample size calculator

Current conversion rate%Minimum detectable effectConfidence%Power%Visitors per week (total)Test

-Visitors per variation

-Total (2 variations)

-Estimated duration

Two-proportion normal approximation, 2 variations (50/50). Tweak the inputs and watch it update live.

A worked example, from test to revenue

A Shopify store converts 2.2% of sessions into orders and wants to detect a 15% relative improvement (from 2.2% to roughly 2.53%). At 95% confidence and 80% power, the math returns 33,284 sessions per variation. At 10,000 sessions a week, that is 47 days, nearly seven weeks, too long for most operations.

Two honest ways out exist, and neither of them is “run it anyway and look early”:

Aim at a larger effect. At 20% relative, the requirement drops to 19,147 sessions per variation, and the same volume closes in 27 days. A larger effect demands a larger change: restructure the buy block, do not reword a label.
Move the metric up the funnel. Deciding on add to cart, with a baseline near 8%, cuts the required sample drastically. The cost is that you measure a step rather than the sale, and you have to confirm the effect on orders afterwards.

Suppose the test ran to 29,000 sessions per variation and finished with 638 orders in control (2.20%) against 734 in the variation (2.53%). Through the same significance engine used across this blog: z = 2.62, p-value approximately 0.0087, with a confidence interval on the difference between +0.08 and +0.58 percentage points. It does not cross zero, so the variation genuinely converts better.

And the revenue: with 40,000 sessions a month and an average order value of 80 dollars, moving from 2.2% to 2.53% is about 132 extra orders, roughly 10,560 dollars in additional revenue per month. Using the lower bound of the interval instead (2.28%), the gain falls to about 2,560 dollars a month. Take both numbers into the meeting: the optimistic one shows the ceiling, the conservative one is what supports an investment decision.

The same win supports two very different projections. Promising the number on the left as if it were guaranteed is the most common reason CRO programmes disappoint three months later.

Shopify specific traps

Apps competing with the test. A mature store accumulates popup, upsell, shipping and review apps, and several of them inject elements into the very page you are testing. If an app shows a popup to 40% of visitors by its own rule, it overlays your experiment unevenly. Map the apps active on the tested surface and disable the ones that interfere during the test.

Theme edited mid test. The Shopify theme editor is inviting, and marketing teams tend to adjust banners weekly. Any edit published mid test changes the page for both variations and invalidates the comparison across that moment. Freeze edits or record the exact date of every change.

Multi currency and Shopify Markets. If the store sells in more than one currency or market, the same test runs across audiences with very different behaviour and basket sizes. Read the result segmented by market before rolling the winner out to everyone.

Bot and app traffic. Part of the traffic reaching your script is not human. Shopify applies its own filters to its dashboard metrics; your testing tool applies its own. Divergence is normal, but if the split between variations drifts from what you configured, that is a sample ratio mismatch and it blocks the reading until you find the cause.

Automatic discounts. A discount rule active during the test can apply unevenly across variations, if the variation changes the cart value and crosses the rule threshold. It is a silent source of average order value difference that has nothing to do with the change being tested.

How long to run, and when to stop

The calculated sample tells you how many sessions you need. It does not tell you when to stop, and those are different questions in a store.

The minimum rule is whole weekly cycles. Monday buying behaviour is not Saturday behaviour, and closing a test on a Thursday after starting on a Monday captures a biased slice of the week. Run in multiples of seven days, even if the sample filled earlier.

The second rule is do not look in order to decide. Opening the dashboard daily and closing as soon as the number looks good is the peeking problem, and it inflates the false positive rate predictably and measurably: the more often you look with intent to stop, the higher the chance of declaring a winner that is not better at anything. Looking to check health (the split is holding, nothing broke, no console errors) is mandatory and carries no such effect; looking to decide is what ruins the test. The article on the peeking problem quantifies the damage.

The third is avoid atypical windows. Black Friday, clearance events, large paid traffic pushes and shopping holidays change purchase intent across the entire audience at once. A result obtained only in that window applies to that window, not to the rest of the year. If the date falls mid test, record the period and read the result with and without it.

Pre-launch checklist

Item	Confirm
Primary metric defined	And compatible with the store volume (see the step 1 table)
Sample and duration calculated	With the calculator above, before switching on, not after
Script at the top of the `head` in `theme.liquid`	With anti-flicker and a short timeout
Validated on a duplicated theme	Variation renders and the conversion event fires on the preview URL
Cart drawer reapplying the variation	Tested by adding a product, not only by loading the page
Competing apps mapped	Popups and upsells on the same surface disabled or documented
Theme edits frozen	Or logged with dates so the period can be excluded from the analysis
Reading segments declared	Mobile and desktop, plus market if there is more than one
SRM check configured	The observed split matches the configured one before any reading

Do this automatically on Donnu

The hard part of A/B testing in a Shopify store is not the installation, it is the discipline: picking a metric compatible with your volume, sizing the sample before switching on, keeping the snippet from delaying the first paint, and reading the confidence interval instead of the day three winner.

Donnu A/B was built for exactly that: a light snippet that never blocks the store, automatic sample sizing and honest Bayesian statistics, with no certainty the data does not support. Start a 14-day free trial and run your first test on the surface where Shopify actually lets you work: everything before checkout.

References

Shopify. Technical documentation on the checkout.liquid layout and its deprecation, including the sunset date of script tags for non-Plus stores. shopify.dev/docs/storefronts/themes/architecture/layouts/checkout-liquid.
Shopify Changelog. “The checkout.liquid theme file is being deprecated”, with the dates of both phases. changelog.shopify.com/posts/the-checkout-liquid-theme-file-is-being-deprecated.
Shopify Partners Blog. “Checkout Extensibility Opens New Ways to Customize Checkouts on Shopify.” shopify.com/partners/blog/checkout-extensibility.
Baymard Institute. Cart Abandonment Rate Statistics. Average across 50 studies of cart abandonment and the distribution of reasons. baymard.com/lists/cart-abandonment-rate.

Frequently asked questions

Can you run an A/B test inside the Shopify checkout?: Not with a generic snippet tool. Shopify sunset checkout.liquid in phases: the Information, Shipping and Payment steps first, then the Thank you and Order status pages along with additional scripts on 28 August 2025. Script tags on those two pages followed the same date for Plus stores and, according to Shopify documentation, are sunset for non-Plus stores on 26 August 2026. Checkout UI Extensions took their place: sandboxed components that Shopify itself validates and renders. Testing inside checkout today means building and shipping an extension, not installing a script. Outside checkout (home, collection, product page, cart) a snippet works normally.
Where do I install the A/B testing script on a Shopify store?: There are two usual paths. The first is editing the theme and placing the script in theme.liquid, as high as possible inside the head, so it decides the variation before the first paint. The second is using an app embed from the Shopify App Store, when the tool offers one. The theme path gives more control over the exact position of the script, which is what prevents the flicker of the original version appearing before the variation.
How much traffic does a Shopify store need for a test to reach significance?: It depends on the current conversion rate and the minimum gain you want to detect. One reference calculated with the math used across this blog: a store converting 2.2% of sessions into orders that wants to detect a 15% relative improvement needs roughly 33,284 sessions per variation. Aiming at 20% relative, the requirement drops to roughly 19,147 per variation. Stores below that volume should test higher in the funnel, where the baseline rate is larger, such as add to cart or checkout entry.
Why do the visitor numbers in my testing tool not match the Shopify dashboard?: Because they count different things. The Shopify dashboard counts sessions using its own window and bot filtering rules; the testing tool counts unique visitors exposed to a variation, with its own identification rule and its own window. A divergence of a few percentage points is normal and expected. What is not normal is the split between variations drifting from what you configured, which is a sample ratio mismatch and needs investigation before any result is read.
Can I test pricing on Shopify with a client-side tool?: Technically you can swap the displayed price with JavaScript, and it is a bad idea: the displayed price would diverge from the real price charged in the cart and checkout, which come from the Shopify backend. Beyond the consumer protection exposure, the test would measure the reaction to a price the store will not charge. Price testing on Shopify belongs in an app that changes the real catalogue price, or in an automatic discount applied by rule, never in a visual override.
Does a Shopify test need to run in whole weeks?: Yes, and it is worth planning for from the start. Buying behaviour on a Monday is not the behaviour on a Saturday, so a test that starts on a Monday and closes on a Thursday captures a biased slice of the week. Run in multiples of seven days even when the calculated sample fills earlier, and if a shopping holiday or a large paid traffic campaign lands inside the window, record the exact dates and read the result with and without that period.