Attributing Revenue to Your A/B Test Winner

When an A/B test ends and the report shows variation B with more total revenue than variation A, the temptation is to call the winner and ship it. The problem is that raw total revenue, on its own, proves no causality: it controls neither for sample size nor for average order value, the two factors covered in the complete GA4 and A/B testing guide as things an analytics tool shows but does not decide for you. This guide focuses on the final step of that process: how to attribute revenue to the right winner of an A/B test, separating what came from the variation from what came from unequal samples, seasonality and high-ticket outliers.

Why “the variation with more revenue” proves nothing on its own

Total revenue is the sum of everything a variation generated in a period. It looks like the most obvious metric to decide a test, because in the end it is what hits the bank account. But total revenue has a structural defect: it grows with the number of visitors, not only with the quality of the variation. Two variations can have the same conversion rate and the same average order value and still show wildly different total revenue, simply because one of them received more traffic in that period (uneven allocation, a delayed start on one side, or a test that ran in phases).

The metric that isolates that effect is revenue per visitor (RPV): total revenue divided by the number of visitors of that variation, over the same period. It is the same denominator as a conversion rate, with money in the numerator instead of a count of events.

A concrete example fixes why this matters. Suppose a checkout test running with uneven traffic allocation between the two variations, a common scenario when a test starts as a gradual rollout:

The table below shows the full numbers behind the chart:

Variation	Visitors	Total revenue	RPV (revenue divided by visitors)
A (control)	6,000	$13,200	$2.20
B	9,000	$17,100	$1.90
Difference	+50% traffic in B	+29.5% in B	-13.6% in B

Variation B took 50% more visitors in the same window, and that alone explains the higher total revenue. Read through revenue per visitor, the picture inverts: A converts better and/or has a better order value per person who passed through the page, even having received less traffic. Without controlling for sample size, any reading of “who made more money” is comparing apples to oranges.

RPV as the primary metric, not gross total revenue

Once RPV is the right denominator, it helps to understand where it comes from. RPV is not a new metric, it is the product of two you already know:

That explains why RPV is the most honest decision metric for any test that touches revenue: a variation can convert worse and still win on RPV if the order value gain compensates the volume loss, and the opposite is equally true. Judging by conversion alone ignores half of the equation; judging by total revenue ignores the other half (sample size). RPV is the only one of the three that answers the right question: “was each visitor who passed through this variation worth more in money?”.

That does not make RPV perfect. It tends to have higher variance than a simple conversion rate, precisely because it carries order value, which is far more dispersed than a binary yes/no event. So RPV should never be read alone, without a significance test behind it. And before computing any RPV, you need to make sure the visitor count per variation comes from the same kind of event on both sides, which is covered in detail in how to track A/B test events in GA4: without a consistent impression event per variation, the RPV denominator is already crooked at birth.

The three denominators, and why teams pick the wrong one

Most revenue attribution arguments are really arguments about the denominator, and nobody says it out loud. There are three candidates, and they answer three different questions:

Denominator	Metric it produces	Question it answers	When it is the right one
Orders	Average order value (AOV)	How much is a purchase worth?	Pricing, bundling and shipping-threshold tests
Converters	Revenue per converter	How much does someone who buys spend?	Post-purchase upsell and cross-sell tests
Visitors exposed to the variation	Revenue per visitor (RPV)	Was this variation worth more per person who saw it?	Almost every A/B test that changes a page

The classic mistake is reporting AOV as if it were the result of the test. A variation that pushes away low-ticket buyers raises AOV and can lower total revenue at the same time, because it converts fewer people. AOV is a diagnostic metric, useful to explain why RPV moved, and a poor decision metric on its own. Whenever a report says “AOV went up 12%”, the next question should be what happened to conversion in the same window.

Isolating the test effect from seasonality

RPV solves the sample size problem, but it does not solve a second, equally common one: comparing variations that ran in different time windows. A frequent mistake is reading variation B revenue in this month report and comparing it against the revenue variation A had before it was replaced, weeks or months ago. That comparison mixes two effects that cannot be separated afterwards: what came from the variation, and what came from the calendar (holidays, payday, a paid media campaign running in parallel, the seasonal cycle of the category).

The rule is easy to state and easy to forget in practice: A and B need to run simultaneously, in the same time window, with traffic split between them at the same time, never in sequence. Only then can any observed RPV difference be attributed to the variation itself instead of to the moment each one ran.

Comparison scenario	What it actually measures	Valid for attributing revenue to the test?
A and B running at the same time, traffic split simultaneously	Effect of the variation, isolated from the calendar	Yes
A last month, B replacing A this month	Effect of the variation mixed with the calendar effect (seasonality, campaigns, holidays)	No
A and B at the same time, but on different traffic sources (A organic only, B paid only)	Effect of the variation mixed with the acquisition channel effect	No, unless traffic source is also controlled
A and B at the same time, same traffic source, same window	Effect of the variation, isolated from calendar and channel	Yes

A detail that shows up often in teams doing a gradual migration from an old page to a new one: if the new variation goes into production replacing the old one instead of running in parallel with it, technically there is no A/B test anymore, there is a before-and-after. Before-and-after can be useful as a directional signal, but it does not let you attribute any revenue difference to the change with confidence, because the calendar is always part of the “after” and never was part of the “before”. This is one of the validity threats covered in common A/B testing mistakes.

The high-ticket outlier problem

Unlike a conversion rate, which is always 0 or 1 per visitor, revenue per order can vary a lot: most orders sit near the typical order value, and a single very large order can pull the mean of one variation up on its own, without representing the typical behavior of anyone who passed through. According to Kohavi, Tang and Xu, in Trustworthy Online Controlled Experiments, revenue metrics usually have heavy-tailed distributions for exactly this reason, and capping (winsorizing) the weight of extreme values is standard practice in experimentation to reduce variance without discarding data. A more recent technical study on monetization metrics in ranking experiments (Pokharna and colleagues, arXiv 2606.04110, 2026) reinforces the point: in revenue metrics, a very small fraction of the highest value users can dominate the variance of an entire test, enough to distort the significance calculation if it is not handled.

An example with real numbers shows the size of the problem. Suppose 12 orders recorded in variation B during the test window, 11 of them near the typical order value of the store and 1 of them an isolated corporate order of $5,400:

The 12 values used here were: $89, $120, $145, $99, $156, $110, $134, $99, $125, $143, $118 and the $5,400 order. The mean of the 12 values is $561.50; the median is $122.50; and the mean recomputed without the $5,400 order falls to $121.64, almost identical to the median. If you used the mean with the outlier to compute the average order value of that variation and project revenue, you would be projecting from a number that no other order in that window comes close to reproducing.

The practical way out, documented both in the Kohavi, Tang and Xu book and in the technical study cited above, is not to discard the large order (it is a real sale, and discarding it without criteria becomes data manipulation): it is to report the median next to the mean, or to cap (winsorize) values above a high percentile (the 99th is a common reference in the experimentation literature) before computing the average order value that feeds the revenue projection. That keeps the order in the conversion count while preventing it from deciding the test on its own.

A second, quieter consequence of the heavy tail is worth naming: it inflates variance, and variance is what significance tests spend sample on. A test whose revenue metric carries a handful of enormous orders needs substantially more traffic to reach the same confidence than the same test read on conversion rate. That is one more reason the workflow below runs the significance test on the conversion rate first, and only then projects money.

Worked example: from significance to annualized revenue

The full path to attributing revenue to an A/B test winner has two stages, and they need to happen in this order: first confirm that the conversion difference is statistically significant (not just sample noise), and only then project what that difference is worth in revenue over a year. Skipping the first stage and jumping to the second is like projecting revenue from a result that may not even be real.

Suppose a checkout test in a store: variation A (control) had 8,000 visitors and 320 conversions; variation B had 8,000 visitors and 376 conversions, running in the same time window, with no relevant order value outliers in that slice.

• Rate of A: 320 divided by 8,000 = 4.00%. Rate of B: 376 divided by 8,000 = 4.70%.
• Relative improvement: (0.0470 minus 0.0400) divided by 0.0400 = +17.5%.
• Pooled rate: (320 + 376) divided by 16,000 = 4.35%.
• Standard error: the square root of 0.0435 x 0.9565 x (1/8,000 + 1/8,000), approximately 0.003225.
• z score: (0.0470 minus 0.0400) divided by 0.003225, approximately 2.17.
• Two-sided p-value, approximately 0.030.

With a p-value of 0.030 (below the 0.05 threshold) and a 95% confidence interval for the difference running from approximately +0.07 to +1.33 percentage points (not crossing zero), the result is statistically significant: variation B converts more, and it is not just small-sample noise. Only now, with significance confirmed, does it make sense to project what that gain is worth in revenue. Check the same calculation in the calculator below, pasting the numbers from your own report:

Control (A)

VisitorsConversions

Variation (B)

VisitorsConversions

Confidence90%95%99%

Control (A) · Rate-

Variation (B) · Rate-

Relative lift-

p-value-

95% CI of the difference-

Two-sided two-proportion z-test. "Not significant" almost always means not enough sample, not that the versions are equal.

With significance confirmed, the next step is the experimentation ROI calculator on this blog, which uses exactly this arithmetic: multiply the conversion rate gain by the average order value and by the projected annual traffic. Assuming this page receives 100,000 visitors per month (1,200,000 per year) and the store average order value is $180, the annualized incremental revenue of replacing A with B is:

The table summarizes both calculations, from the significance test to the revenue projection:

Step	Input	Result
Significance (two-proportion z-test)	8,000/320 (A) vs 8,000/376 (B)	z = 2.17 · p = 0.030 · significant · B wins
95% confidence interval of the difference	same data as above	+0.07 to +1.33 percentage points
Annualized incremental revenue	traffic 1,200,000/year · AOV $180	$1,512,000 per year

Notice that the projected incremental revenue uses the conversion rate difference, not the total revenue observed during the test: that is what makes the projection immune to whatever sample size the test happened to have. The test ran with 8,000 visitors per variation to prove the effect is real; the $1,512,000 projection applies that proven effect to the real traffic of the funnel, not to the small traffic of the experiment.

Report the range, not the point estimate

One number deserves a warning label before it reaches a slide. The $1,512,000 comes from the point estimate of the lift (+0.70 percentage points). The confidence interval says the true effect plausibly sits anywhere between +0.07 and +1.33 percentage points. Running the same projection on both bounds gives the honest range:

Both bounds come from the same formula, only swapping the absolute lift: 1,200,000 x 0.0007 x 180 is about $151,000 per year, and 1,200,000 x 0.0133 x 180 is about $2,873,000 per year. This is not a reason to distrust the test, it is a reason to present the result as a range and to keep collecting data when the decision downstream depends on the size of the effect and not only on its direction. A result that is significant tells you the direction is real; only a narrow interval tells you the magnitude is dependable.

Common revenue attribution mistakes

Mistake	How it shows up	Fix
Deciding by total revenue	“B made $4,000 more, ship it”	Compare RPV, which normalizes by visitors
Comparing across time windows	A last month vs B this month	Run both simultaneously, same window, same sources
Letting one big order decide	Mean order value jumps 4x in one variation	Report median next to mean, winsorize above a high percentile
Projecting before confirming significance	Annual revenue slide built on p = 0.21	Confirm significance first, project second
Reporting the point estimate alone	“This test is worth $1.5M a year”	Report the range implied by the confidence interval
Using AOV as the decision metric	“AOV rose 12%, the test won”	AOV explains why RPV moved; RPV decides

Make this automatic with Donnu

Everything this guide described (separating RPV from total revenue, making sure both variations run in the same time window, handling order value outliers sensibly, and projecting revenue only after significance is confirmed) is manual work that most teams redo from scratch on every test, and it is exactly where revenue attribution goes wrong: deciding too early, with the wrong number. In Donnu, traffic is split between variations in the same time window by construction (there is no “run A one month and B the next”), and the honest significance calculation is available as soon as the sample allows, so you never confuse total revenue with revenue per visitor, nor celebrate a gain before it is statistically real.

Start a free trial and let simultaneous traffic splitting and the significance calculation be the tool job, while you focus on deciding with the right number: RPV, not total revenue. For the full picture of how analytics and A/B testing fit together, see the complete GA4 and A/B testing guide.

References

• Optimizely. Revenue per visitor (definition, RPV as the product of conversion and order value, and the warning about metric noise). optimizely.com/optimization-glossary/revenue-per-visitor.
• Kohavi, R., Tang, D. & Xu, Y. Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press, 2020. Companion material at experimentguide.com.
• Pokharna, H. et al. Variance Reduction for Heavy-Tailed Monetization Metrics in Ranking Experiments via Post-Stratification. arXiv, 2026. arxiv.org/html/2606.04110.
• Kohavi, R. & Thomke, S. The Surprising Power of Online Experiments. Harvard Business Review, 2017. hbr.org/2017/09/the-surprising-power-of-online-experiments.

Read also: GA4 and A/B testing: the complete integration guide · How to track A/B test events in GA4 · Common A/B testing mistakes · Revenue per visitor calculator · Leia em português