A/B Testing on WooCommerce: The Complete Guide
WooCommerce a/b testing: page caching, blocks vs shortcode checkout, which metric fits each stage and the sample math that decides what you can test.
📚 This article is part of the guide Ecommerce Checkout Optimization: The A/B Testing Playbook.
A/B testing on WooCommerce is no different from A/B testing on any other store, except for four details that only show up on a WordPress install and that sink most first attempts: page caching, the coexistence of the block checkout and the classic checkout, plugin conflicts, and the choice of which stage to measure. This guide is part of the ecommerce checkout optimization playbook and covers all four, shows which metric fits each stage of the store, and gives you the math that decides what you can and cannot test at your traffic level, with a live calculator and an end-to-end worked example.
What is different about a WooCommerce store
WooCommerce is an ecommerce plugin for WordPress, with more than 7 million active installations according to the official WordPress plugin repository (wordpress.org). It ships no experimentation engine, so the test always comes from outside, through one of two paths:
| Path | How it works | Advantage | Main WordPress trap |
|---|---|---|---|
| Client-side snippet | A tag in the header swaps content in the visitor browser | Installs in minutes, no deploy needed | Flicker if the tag loads late; depends on JavaScript |
| Server-side in PHP | A hook assigns the variation before the page is assembled | No flicker, works with JavaScript disabled | Page caching can freeze one variation and serve it to everyone |
The difference between the two models is detailed in the client-side vs server-side A/B testing guide. For most WooCommerce stores, the client-side path is the only one that ships in a short timeframe, and the next section explains why.
Obstacle 1: page caching
This is problem number one, and it is silent. Almost every serious WordPress store runs page caching, either through a plugin or through a CDN. Caching exists precisely so PHP does not run on every visit: it stores the HTML generated once and returns that same file to the next people.
That is excellent for performance and fatal for a naive server-side test. If your code assigns the variation in PHP and the cache stores the first response generated, everyone starts receiving that variation. The test keeps “running” in the dashboard, and the real traffic split is no longer the one you configured.
Correct server-side testing can coexist with caching: the cache key has to include the variation, or the decision has to happen at the edge, before the cache responds. That is infrastructure work, described in the server-side A/B testing implementation guide. The mistake is running the test without handling the cache and then trusting the report.
Whichever path you choose, one check is mandatory on day one: look at the real visitor split between variations. A split that should be 50/50 and shows up as 68/32 is not bad luck, it is an instrumentation defect, and the whole test stays invalid until it is fixed. That check has a name, sample ratio mismatch, and it is covered among the most common A/B testing mistakes.
Obstacle 2: block checkout vs classic checkout
According to WooCommerce documentation, stores started from the release of version 8.3, in November 2023, ship the Cart and Checkout blocks as the default experience with no configuration required (WooCommerce, Cart and Checkout Blocks as Default). Older stores normally still run the classic checkout, assembled by shortcode.
For testing, the consequence is direct: the two versions generate different HTML. A CSS selector written against the classic checkout simply finds nothing on the block checkout, and the test runs without ever applying the variation, producing a perfect tie that looks like a result and is a defect.
Before writing any variation, check which of the two your store uses and write the selector against the real page structure. If you manage more than one store, do not reuse selectors between them without checking.
Obstacle 3: plugin conflicts and logged-in visitors
Two WordPress specifics close the list of technical traps.
The first is the plugin set. A typical WooCommerce store runs dozens of them, and several touch the same place your test touches: checkout optimizers, custom field plugins, shipping plugins, popup plugins. When two pieces of code alter the same element, whoever applies last wins, and the outcome varies with load order. The symptom is a variation that “sometimes shows up”.
The second is the logged-in visitor. Administrators, editors and returning logged-in customers usually see the site with an admin bar, caching disabled and sometimes different prices. Including that audience contaminates both sides unevenly. The practical rule is to exclude logged-in users with administrative roles from the experiment, and to consciously decide whether logged-in customers are in or out, documenting the choice.
The math that decides what you can test
Here is the part that settles the most common merchant objection: “my store is too small to test”. Almost always the sentence is wrong, and what is wrong is the stage chosen for measurement.
Consider a store with 6,000 visits per week and an overall conversion rate of 1.8% (visit to order). To detect a 20% relative gain at 95% confidence and 80% power, the required sample is 23,507 visits per variation, which takes 55 days.
Two-proportion normal approximation, 2 variations (50/50). Tweak the inputs and watch it update live.
Set the calculator above to base rate 1.8, minimum detectable effect 20 (relative) and 6,000 visitors per week to reproduce that number. Nearly two months is a timeline most stores will not accept, and that is where the team either gives up or, worse, stops the test halfway.
Now change the stage you measure. The same store gets 1,200 checkout starts per week, and 45% of them become orders. Testing a change inside checkout, measuring completion rate from checkout start, with a 10% relative gain, requires only 1,931 visitors per variation, which takes 23 days.
Two-proportion normal approximation, traffic split evenly across variations. The date uses your timezone and updates live.
| Design | Base rate | Volume that feeds it | Sample per variation | Duration |
|---|---|---|---|---|
| Whole store, visit to order, 20% gain | 1.8% | 6,000 visits/week | 23,507 | 55 days |
| Checkout stage, start to order, 10% gain | 45% | 1,200 starts/week | 1,931 | 23 days |
The difference is not statistical magic, it comes from two things at once: high base rates need far smaller samples for the same relative gain, and the funnel filters for people who have already shown intent. Testing inside checkout is the lowest sample cost path in almost every small and mid-sized store.
The honest counterweight: measuring the stage is not the same as measuring the business. A change that improves checkout completion can, for example, raise the rate and lower average order value if it pushes customers to finish faster. That is why revenue per visitor is always a guardrail, even when the primary metric is the stage.
A worked example you can reproduce
Suppose that checkout test ran to the full 1,931 visitors per variation. The classic checkout (control) closed 869 orders, a completion rate of 45.00%. The variation with fewer fields closed 949 orders, 49.15%.
Running those four numbers through the same significance engine this blog uses: z = 2.579, p-value about 0.0099, with a 95% confidence interval for the difference between +1.00 and +7.29 percentage points, and an observed relative improvement of +9.2%. The interval does not cross zero, so the gain is real.
That width is the honest reading of a small sample. A 1,931 per variation test answers “did it help” with confidence and answers “by how much” only within a wide band. If your decision depends on the size of the gain and not just its sign, the statistical significance guide explains how to read the interval instead of the point estimate.
Which metric to use at each store stage
| Where the change lives | Primary metric | Guardrails |
|---|---|---|
| Home and category pages | Product page visit rate | Revenue per visitor, exit rate |
| Product page | Add-to-cart rate | Revenue per visitor, return rate when available |
| Cart | Checkout start rate | Average order value, revenue per visitor |
| Checkout | Completion rate from start | Revenue per visitor, payment error rate |
| Price, shipping and promotions | Revenue per visitor | Margin, average order value, conversion rate |
The rule behind the table: the closer the primary metric sits to the change, the faster the test closes, and the further it sits from money, the more indispensable the guardrail becomes. Product page and cart get detailed treatment in the product page A/B testing guide and in the cart abandonment guide.
Step by step for your first test
- Pick the stage with the largest absolute loss. Add up how many people leave at each funnel step and attack the step that loses the most people, not the one that looks ugliest.
- Write the hypothesis before opening the tool. What changes, why, which primary metric, what minimum effect would justify shipping it.
- Size before running. Calculate sample and duration with the real traffic of that stage. If the answer is longer than your acceptable window, change the ambition or the stage, never the rigor.
- Install the tag and check the real split. On day one, confirm the distribution matches what you configured. Check on mobile too, and logged out, in a private window.
- Let it run in whole weeks. Stores have strong day-of-week seasonality. Ending mid-week compares Monday against Saturday.
- Read the confidence interval first. An isolated point estimate becomes a promise the data cannot keep.
- Ship the winner properly in the theme or plugin. A variation served by snippet is temporary; the permanent gain only exists once it becomes store code.
Common WooCommerce A/B testing mistakes
| Mistake | Warning sign | Fix |
|---|---|---|
| Ignoring the cache in a server-side test | Traffic split far from configured | Vary the cache key per variation, or go client-side |
| Writing the selector for the wrong checkout | Perfect tie and a variation that never appears | Check whether the store uses blocks or the classic shortcode first |
| Including logged-in administrators | Few sessions with very atypical behavior | Exclude administrative roles and document the call on logged-in customers |
| Measuring only store-wide conversion in a small shop | Tests that run for months and get abandoned midway | Measure the stage, with revenue per visitor as guardrail |
| Ending mid-week | A result that flips sign depending on the reading day | Run in complete weekly blocks |
| Testing during a big promotion | Completely different buying behavior | Avoid atypical periods or treat them as a separate test |
| Running three tests in the same funnel at once | Results that do not add up once shipped | One test per stage, or a factorial design declared upfront |
| Leaving the winning variation in the snippet forever | The gain depends on the tool continuing to load | Implement it in the theme or plugin and remove the test |
Make this automatic with Donnu
The hard part of testing on a WooCommerce store is almost never the idea, it is the operation around it: confirming the traffic split came out right and reading the result without turning an imprecise estimate into a revenue promise. Donnu covers exactly that stretch: you paste a snippet that does not block the store from loading, define the metric, and Donnu tracks the real traffic split, warns you when it drifts from what you configured, and returns the verdict with the 95% confidence interval up front, without declaring a winner before the variation accumulates at least 200 visitors and 7 days live.
Start a 14-day free trial and run your first test on the step that loses the most people. For the full picture, see the checkout optimization playbook and the list of A/B testing tools for WordPress.
References
- WooCommerce. Cart and Checkout Blocks as Default. Official documentation. woocommerce.com/document/cart-checkout-blocks-status.
- WordPress.org. WooCommerce plugin page in the official repository. wordpress.org/plugins/woocommerce.
- Google Search Central. A/B testing best practices for Search. developers.google.com/search/docs/crawling-indexing/website-testing.
- Kohavi, R., Tang, D. and Xu, Y. Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press, 2020. Companion material at experimentguide.com.
Read also:
Frequently asked questions
- Does WooCommerce have built-in A/B testing?
- No. WooCommerce is an ecommerce plugin for WordPress and ships no experimentation engine, so the test always comes from outside: a client-side snippet that swaps content in the browser, or a server-side implementation in PHP that assigns the variation before the page is built. Each path has its own trap on a WordPress install, and the biggest one by far is page caching.
- Why does page caching break A/B tests on WooCommerce?
- Because most WordPress stores serve pre-generated HTML from a caching plugin or a CDN, so a server-side test that picks the variation in PHP can have that response frozen and handed to everybody. The classic symptom is a traffic split far away from the 50/50 you configured. Client-side tests do not suffer from this, because the HTML is identical for everyone and the swap happens in the browser afterwards.
- Can I test the WooCommerce checkout page?
- Yes, but the selector target depends on which checkout your store runs. According to WooCommerce documentation, stores created from version 8.3 onward, released in November 2023, ship the Cart and Checkout blocks as the default experience, while older stores usually still run the classic shortcode checkout. The two produce different HTML, so a selector written for one silently finds nothing on the other.
- My store has low traffic. Can I still test?
- It depends on which stage you choose to measure. Store-wide conversion has a low base rate and needs a large sample: in the worked example below, detecting a 20% relative gain on a 1.8% base takes 55 days at 6,000 visits per week. Testing inside checkout, where the base rate is far higher, closes in 23 days with only 1,200 checkout starts per week. Changing the stage you measure changes feasibility without giving up any rigor.
- Which metric should a WooCommerce store test use?
- The primary metric has to sit as close as possible to what the change actually alters, and revenue per visitor should always be among the guardrails. A product page change calls for add-to-cart rate as primary; a checkout change calls for order completion rate from checkout start. In neither case can revenue per visitor be left out, because it is the metric that exposes a volume gain bought with a smaller basket.
- Do I need a plugin to run A/B tests on WooCommerce?
- Not necessarily. Snippet-based tools work by pasting a tag in the site header, which can be done through a code insertion plugin, the theme, or your tag manager. A dedicated plugin makes installation easier but adds one more item to the store plugin set, and plugin conflict is one of the most common causes of strange behavior on a WordPress install.