Budget Pacing: When the Daily Cap Breaks Your Test

A daily budget cap does not only cut spend, it cuts hours. A test arm that burns through its budget by 3 pm is not a random sample of the day: it is a sample of the hours when budget still existed. Because conversion rate varies by hour, the throttled arm arrives in the report with a different traffic mix than its counterpart, and the measured difference now carries time of day bundled together with the page effect. In this guide worked example, a treatment that was truly 5.0000% better reads as 15.3226% worse, with a p value of 0.008007 and a confidence interval sitting entirely on the negative side, which is to say it has every outward sign of a trustworthy result. The signal that exposes the problem is not the p value, it is sample ratio mismatch. This guide covers the mechanism, the caps Google and Meta document, the worked example with all three possible readings, how much power survives the fix, and why pacing itself cannot be tested by randomizing users. This guide is part of our complete guide to A/B testing.

A daily cap cuts hours, not just spend

A daily budget almost always reads as a constraint on money. For anyone measuring, it is a constraint on sampling, and that is where the trouble starts.

When an arm hits its cap, it does not serve a little less in every hour. It stops serving from a certain moment onward. The arm does not end up with a shrunken version of the whole day, it ends up with the first part of it. If conversion rate were flat across all 24 hours, that would cost nothing but sample size. It is not flat: morning traffic and late afternoon traffic come from different intents, different devices and different stages of the purchase decision.

What you get is a textbook confound, and it is structural rather than statistical: the variable “arm” is now correlated with the variable “hour of day”. No p value adjustment repairs that, because it is not noise. It is bias, and it has a predictable direction, always favouring the arm that serves all day whenever the lost hours are the high rate ones.

It is worth separating two failures that look alike. An arm that gets less traffic uniformly across every hour only costs statistical power: the comparison stays honest, it just gets less sensitive. An arm that gets the same traffic up to a certain hour and nothing after costs validity. A daily cap produces the second situation, and that is the one that flips verdicts.

The caps the platforms actually document

Before any arithmetic, it helps to know that a daily budget is not a hard ceiling on either major platform. Both document explicitly that a single day can spend more than the number you typed, and each one compensates differently.

Google Ads states that on a given day, a campaign might spend up to twice the average daily budget to take advantage of fluctuations of traffic, and that at the end of the month you will have spent no more than 30.4 times the average daily budget. The documentation adds a detail that muddies reporting: in some circumstances the served costs, meaning the cost of all the clicks or impressions the campaign received, may exceed the daily or monthly spending limits, but the advertiser never actually pays more than those limits. In other words, the number in the delivery report and the number on the invoice can diverge, and for measurement purposes it is the first one that matters.

Meta works on a fixed weekly window instead. The documentation says the daily budget tells Meta an approximate amount to spend per day over the course of the week starting Sunday and ending Saturday. On days when better advertising opportunities are available, Meta may spend up to 75% over the daily budget. With ad set budget sharing turned on, it may additionally share up to 20% of the flexible daily budget with other ad sets, which works out to as much as 210% of the daily budget. The stated ceilings: for every day ending at midnight, spending will not exceed 175% of the daily budget; for every week ending Saturday at midnight, it will not exceed 7 times the daily budget.

platform	budget entered	one day ceiling	period ceiling	compensation period
Google Ads	$500 per day (average)	$1,000 (2 times)	$15,200.00 per month (30.4 times)	calendar month
Meta	$100 per day	$175 (175%)	$700 per week (7 times)	Sunday to Saturday
Meta, with ad set budget sharing	$100 per day	$210 (210%)	$840 per week	Sunday to Saturday

Meta documentation on budget sharing spells out the per ad set formulas: maximum spend on a day is the daily budget plus that day shared budget, multiplied by 1.75; maximum spend over the Sunday to Saturday week is the same sum multiplied by 7. It also states something that matters for testing: at a campaign level, spend remains the same whether budget sharing is on or off. Sharing redistributes between ad sets, it does not raise the total.

Three practical consequences follow.

The first is that the ceiling does not protect you from the problem in this guide, it makes it worse. If an arm can spend 175% of its budget on a given day, it runs dry earlier on the days the platform decides to accelerate, and its cutoff hour starts moving from one day to the next. An arm whose stopping time drifts across the week has an hourly mix that drifts with it.

The second is a calendar problem. Meta budget week is fixed, Sunday through Saturday. A test that starts on a Wednesday runs its first week under a different compensation rule than its second, because the first calendar week of the test straddles two budget weeks. That is a period effect, and it is confounded with the treatment effect. Start tests on the boundary of the platform budget week, not in the middle of it.

The third is that Google already discards data over this by default. According to the platform experiments FAQ, the first 7 days of data is discarded to account for the experiment ramp-up time. That is a design decision made by the platform rather than the advertiser, and it exists precisely because the start of a media experiment is not comparable with its middle.

Worked example: a winning treatment that reads as a loser

The numbers below came out of the same statistics engine that powers the calculators on this page.

The design: a landing page test with two arms running on paid media. Arm B was given a lower daily budget and exhausts it at 3 pm; arm A serves all day. The treatment applied to B is genuinely 5% better in relative terms than A rate, in every hour of the day. There is no hour interaction in the treatment at all: it is uniformly positive.

Base rates by hour band are identical for both arms before the treatment is applied:

hour band	base rate	note
midnight to 3 pm	1.0000%	browsing traffic
3 pm to midnight	1.6000%	decision traffic, 1.6 times the morning rate

The counts the test window produced:

arm	midnight to 3 pm	3 pm to midnight	total
A (uncapped)	360 of 36,000	384 of 24,000	744 of 60,000 = 1.2400%
B (throttled at 3 pm)	378 of 36,000	nothing served	378 of 36,000 = 1.0500%

Notice that in the band where both arms served, B converts better: 1.0500% against 1.0000%, which is exactly the 5% relative treatment. The effect is sitting right there, intact, in the only band where the comparison is legitimate.

Now the three possible readings of that same experiment, verbatim from the engine:

reading	A	B	abs. diff.	rel. lift	z	p	95% CI (pp)	verdict
naive, full window	744/60,000 = 1.2400%	378/36,000 = 1.0500%	-0.1900 pp	-15.3226%	-2.6518	0.008007	-0.3276 to -0.0524	A wins, significantly
correct, midnight to 3 pm only	360/36,000 = 1.0000%	378/36,000 = 1.0500%	+0.0500 pp	+5.0000%	0.6660	0.505404	-0.0971 to +0.1971	flat
counterfactual, B uncapped	744/60,000 = 1.2400%	781/60,000 = 1.3017%	+0.0617 pp	+4.9731%	0.9536	0.340311	-0.0651 to +0.1884	flat

The naive reading does not merely get the size wrong. It gets the sign wrong and is significant while doing so. A p value of 0.008007 with the whole confidence interval on the negative side looks exactly like a result you can act on, and the true effect is positive. Put that report in front of a decision meeting and the better page gets killed with convincing statistical backing.

To reproduce the table, enter 744 conversions out of 60,000 visitors against 378 out of 36,000 and the screen shows a -15.3% lift at p = 0.0080. Then switch to 360 out of 36,000 against 378 out of 36,000 and it shows +5.0% at p = 0.5054. Those are the table values, rounded. The same significance calculator runs both readings.

Control (A)

VisitorsConversions

Variation (B)

VisitorsConversions

Confidence90%95%99%

Control (A) · Rate-

Variation (B) · Rate-

Relative lift-

p-value-

95% CI of the difference-

Two-sided two-proportion z-test. "Not significant" almost always means not enough sample, not that the versions are equal.

The arithmetic behind the inversion is a weighted average, and it is simple enough to fit on one line. Arm A carries 60.00% of its traffic in the 1.0000% band and 40.00% in the 1.6000% band:

36,000 x 1.0000% + 24,000 x 1.6000% = 360 + 384 = 744, and 744 / 60,000 = 1.2400%

Arm B carries 100.00% of its traffic in the weaker band. The 1.2400% figure for A is not A rate in any hour of the day: it is a blend of 1.0000% and 1.6000%, weighted by how much volume each band contributed. Comparing 1.2400% with 1.0500% compares two different blends of the clock, with the page itself entering as a rounding detail.

The third row of the table closes the argument. In the counterfactual where B has no cap and serves the same 60,000 visits, it produces 781 conversions, a rate of 1.3017% against 1.2400% for A, a lift of 4.9731%. Remove the cap and the experiment returns the true effect. Every bit of the damage in the first row is the cap talking.

Sample ratio mismatch is the alarm, not the p value

The obvious question is how anyone catches this before making the wrong call. The answer is that the signal exists, it is cheap to compute, and it has nothing to do with the primary metric.

The test was designed to split traffic evenly. It came out 60,000 against 36,000, meaning 62.50% of traffic landed in a single arm instead of the intended 50%. A goodness of fit test on that split returns a z of 77.46, with a p value below 1 divided by 10 to the power of 300. There is no subtlety left to debate: a split like that does not happen by chance.

The working rule is deliberately blunt: no test with a sample ratio mismatch should be read, no matter how good the p value on the primary metric looks. The 0.008007 in the first row is not evidence for the conclusion, it is evidence that the bias is large. The stronger the imbalance, the more confident the wrong number looks. The mechanism and the diagnostics are laid out in sample ratio mismatch.

It is worth being precise about why the imbalance shows up here. There was no randomization failure, no instrumentation bug, no bot traffic. The randomization worked perfectly: every visitor who entered an auction had an equal chance of landing in either arm. What happened is that one arm stopped entering auctions after 3 pm. The imbalance is a downstream consequence of the budget, and that is exactly why it is such a good detector for this particular failure: the cause and the symptom are the same event.

How much power survives the clean window

Fixing the bias is easy: compare only the hours in which both arms served. What almost nobody computes is the price of that fix.

Sizing the correct reading, on a 1.00% baseline, two sided at 95% and 80% power:

relative MDE	N per arm	duration at 252,000 visitors per week
5.0000%	637,010	36 days
10.0000%	163,095	10 days
15.0000%	74,193	5 days

With the 36,000 per arm the clean window left behind, power to detect the true 5% is 9.78%. Put differently, even with the bias removed, that test had roughly a one in ten chance of finding an effect that was genuinely there. At 420,000 per arm power climbs to 62.36%, still short of the conventional 80%. Reaching 80% would take 637,010 per arm, nearly eighteen times what the clean window delivered.

The daily cap charges you twice: first it flips the verdict, then it destroys the power of the only window that was still usable. That pairing is especially nasty because each charge hides behind the other. Anyone who fixes the bias without re-running the power calculation trades a confident false positive for a quiet false negative, and concludes that the new page “made no difference”.

To reproduce the first row, enter a 1% baseline, a relative minimum detectable effect of 5%, 95% confidence and 80% power in the sample size calculator: the answer is 637,010 per arm.

Current conversion rate%Minimum detectable effectrelative (%)absolute (pp)Confidence909599%Power8090%Visitors per week (total)Testtwo-sidedone-sided

-Visitors per variation

-Total (2 variations)

-Estimated duration

Two-proportion normal approximation, 2 variations (50/50). Tweak the inputs and watch it update live.

The most useful way to read that table is as planning, not autopsy. If the effect you expect is on the order of 5% relative on a 1% baseline, the test needs more than 1.2 million visitors across both arms. A design that lets one arm lose 40% of its traffic to a budget cap is burning an enormous share of its sample budget before it measures anything at all. The same reasoning shows up in how many visitors an A/B test needs and in minimum detectable effect.

Budget pacing itself cannot be tested by randomizing users

So far pacing has played the villain. It is worth flipping the question: what if pacing is the thing you actually want to measure?

Then the design changes completely, and the reason is stated in the literature. The authors of the LinkedIn budget pacing algorithm state that a per user A/B test will not do for evaluating pacing. Their argument is direct: the pacing algorithm affects a campaign budget spending for an entire day, so the same treatment has to be applied to that campaign for the entire day. Splitting traffic into two buckets is not enough, because the mechanism does not live in the user, it lives in the campaign and the day.

The natural alternative, splitting campaigns into two groups, does not close the gap either. It measures the advertiser benefit but not the revenue benefit to the publisher, because campaigns from both groups compete in the same auction. To see the whole effect, the algorithm has to be on for every campaign for the whole day.

Their stated solution was to run pacing on alternate days. That is a switchback design: the randomization unit becomes the day rather than the user or the campaign, and every unit receives both treatments over the course of the experiment. The price is the usual one for that design: few units, and day of week effects to control for.

It is worth noting why they went to that trouble. The paper starts from the observation that a generalized second price auction, applied greedily to every impression, is suboptimal for both advertiser and publisher when the advertiser has a finite budget: high performing campaigns burn through their budget and drop out of the market early. That is precisely the mechanism in this guide, seen from the platform side.

Their chosen campaign lifetime metric is telling: time to exhaust 95% of budget, reported as a median in hours. The results on the Direct Ads product, over 2 weeks in the fourth quarter of 2013, with no sampling:

metric	without pacing	with pacing
median campaign lifetime	13.54 hours	19.50 hours
unique impressions per spend	baseline	+7.74%
number of campaigns served	baseline	+4.74%
cost per request	baseline	+5.67%
over delivery	3.8%	3.4%
unique campaigns seen per member	baseline	+0.15%

Median campaign lifetime rose 44%, from 13.54 to 19.50 hours. Translated into the problem in this guide: without pacing, the median campaign died before it completed 14 hours of the day, which is to say before 3 pm if it started at midnight. The scenario in the worked example is not a laboratory construction, it is roughly the median of a real system before it got a pacing algorithm.

The within day price effect is the other side of the same coin. With pacing on, cost per click sits lower early in the day and higher for a considerably longer stretch through the second half, because the algorithm deliberately throttles high performing campaigns early on, campaigns that would otherwise all compete at once and drain their budgets fast. Anyone measuring conversion by hour band needs to know this: the price of traffic is not flat across the day, and pacing flattens it on purpose.

The operational parameters they report give a sense of how granular this kind of system is: the pass-through rate starts at 10% for every campaign, the adjustment rate is a constant 10%, and the system updates roughly 12% of all campaigns every 7 seconds. A mechanism that adjusts on a seconds long cadence is not something a daily report can observe properly.

Where this ends and platform splits and Smart Bidding begin

Three guides on this blog cover adjacent ground, so it helps to draw the lines.

The randomization mechanism inside the platform belongs to ad platform split testing. How the platform divides users, whether the split is cookie based or per search, how it keeps one person inside a single arm: all of that comes before the problem in this guide.

A treatment that changes shape over the window belongs to Smart Bidding. There the issue is that the treated arm on day 3 is not the treated arm on day 30, because the optimizer recalibrated.

Here the issue is different and blunter: the arm stops serving partway through the day. It is not a change of shape, it is an absence. And the configuration choice that decides whether you fall into it at all comes before either of the other two.

The Google Ads experiments FAQ describes that choice explicitly. For certain Search, App and Display experiments, Google supports budget splits, while for some others, such as Performance Max, Demand Gen and Video, it supports a traffic split. The difference matters a great deal here:

split type	what the platform divides	throttling risk
budget split	the total budget across the arms, for example 70% to control and 30% to treatment	high: the 30% arm hits its ceiling long before the 70% arm does
traffic split	the traffic, meaning how many times the ads enter auction	low: both arms sit under the same budget constraint

The documentation also describes how the user split works: it occurs at the start of the experiment and Google systems try to ensure fairly balanced arms based on the probability of entering an auction, using cookie based or user based splits. Note the wording: balance on the probability of entering an auction, not on impressions actually served. If one arm runs out of budget, entry balance is still honoured while exit balance collapses. That gap is exactly what sample ratio mismatch catches.

A 70 to 30 split is a recipe for the failure in this guide whenever the two halves do not share the same exhaustion profile. If you have the choice, prefer a traffic split. If only a budget split is on offer, keep the division symmetric and watch both arms stopping times every day.

Two calendar notes finish the boundary. Google recommends running the experiment for at least 4 to 6 weeks, or longer if you have a long conversion delay, which interacts with the mechanism described in conversion lag. And it discards the first 7 days of data to account for ramp-up time. Long windows help against throttling in one sense, by diluting odd days, and hurt in another, by giving someone more opportunity to touch the budget mid-flight.

When the effect you care about is account wide, or when the arms would compete in the same auction regardless, the answer is to stop using the platform as the unit: that design lives in geo experiments. And when the question is which budget level maximises return rather than which page converts better, the two companion guides in this batch cover it: budget saturation deals with the response curve and the point where spending more stops paying, and budget reallocation deals with moving money between channels without being fooled by correlation.

Guardrail metrics

None of these is the primary metric of the test, and all of them should be checked before it. If any one of them fires, the primary metric is not interpretable.

signal	what to watch	what to do if it fires
timestamp of last event served	per arm and per day, the hour of the last recorded event	if one arm systematically stops earlier, it is capped; do not read the primary metric
share of traffic by hour band	each arm fraction of traffic in each hour band, compared across arms	any material divergence invalidates the full window comparison; trim to the intersection
limited by budget status	the limitation flag the platform exposes, per day and per arm	fix the budget and restart the window; a throttled arm is not a treated arm
visitor ratio between arms	deviation from the planned split, with a goodness of fit test	above your mismatch threshold, stop the analysis and find the cause
daily spend against the documented ceiling	how many days hit 2 times the daily budget on Google, or 175% of it on Meta	ceiling days are early stopping days; flag them and check whether they hit only one arm
over delivery	the fraction of spend above plan, per arm	asymmetry between arms means the platform is compensating differently in each one

The rule that covers all six: before comparing rates, confirm both arms were live during the same hours.

Common mistakes

• Reading the full window when one arm stopped mid day. This is the worked example. It inverts the sign of the effect and hands you a convincing p value while it does so.
• Treating a good p value as proof the test is healthy. A p value measures noise, not bias. Under a large sample ratio mismatch, the more bias there is, the better the p value gets.
• Giving the two arms different budgets because one is “just the test”. An asymmetric budget is an asymmetric stopping time, which is an asymmetric traffic mix. If the platform only offers a budget split, split it down the middle.
• Choosing the analysis window after seeing the numbers. Trimming to the overlapping hours is correct, and it has to be written into the plan. The same trim chosen after the fact inflates false positives exactly the way stopping early does.
• Fixing the bias and skipping the power recalculation. After the trim you are left with 36,000 per arm and 9.78% power. Concluding “no difference” at that power asserts something the test was never able to show.
• Starting the test in the middle of the platform budget week. Meta week runs Sunday to Saturday. A test that starts on a Wednesday has its first week split across two compensation periods.
• Assuming the daily budget is a hard ceiling. Google documents up to 2 times the daily budget on a given day; Meta, up to 175% of it, or 210% with ad set budget sharing. Stopping times drift from day to day because of that.
• Trying to test pacing by randomizing users. The treatment acts on a campaign for the whole day. A user split measures something else, and a campaign split leaves the auction effect out.

Make this automatic with Donnu

The deeper fix is to take the sampling decision out of the media system entirely.

When randomization happens on your own page, after the click, the daily budget no longer decides who lands in which arm. It still decides how much traffic arrives and at what hours, but that falls on both arms equally, because both arms live inside the same traffic. It is the difference between a ceiling that biases and a ceiling that merely shrinks the sample, and only the second one is recoverable with more time.

With Donnu the experiment runs on your own site, with the analysis window and the trim rule declared before the test goes live, and the report shows the visitor ratio between arms next to the difference in rates, which is the order those two numbers need to be read in. Donnu is one option among several, and what does not change from tool to tool is the principle: if one arm stopped serving during hours the other one served, the number you read is not the treatment effect, it is the clock.

Frequently asked questions

The questions at the top of this page cover whether a daily cap breaks an A/B test, why a throttled arm looks worse than it is, how to spot an arm that hit its ceiling, whether trimming the window solves it, whether pacing itself can be tested, and how far above the stated daily budget a platform can actually spend.

References

• Agarwal, D., Ghosh, S., Wei, K. and You, S. (LinkedIn). Budget Pacing for Targeted Online Advertisements at LinkedIn. KDD 2014, pp. 1613-1619. Source of the claim that a per user A/B test will not serve to evaluate pacing, because the algorithm affects a campaign budget spending for an entire day; of the argument that splitting campaigns measures the advertiser benefit but not the publisher benefit, since campaigns from both groups compete in the same auction; of the decision to run the algorithm on alternate days; of the campaign lifetime metric defined as time to exhaust 95% of budget; of the results table on the Direct Ads product over 2 weeks in the fourth quarter of 2013 (median lifetime from 13.54 to 19.50 hours, plus 7.74% unique impressions per spend, plus 4.74% campaigns served, plus 5.67% cost per request, over delivery from 3.8% to 3.4% and plus 0.15% unique campaigns per member); of the description of the within day cost per click curve; and of the operational parameters of a 10% initial pass-through rate, a constant 10% adjustment rate and roughly 12% of all campaigns updated every 7 seconds. Checked on September 25, 2026. dl.acm.org.
• Google Ads Help. Average daily budget. Source of the statement that on a given day a campaign might spend up to twice the average daily budget to take advantage of fluctuations of traffic, that at the end of the month spend will be no more than 30.4 times the average daily budget, and of the caveat that served costs may exceed the daily or monthly spending limits without the advertiser ever paying more than those limits. Checked on September 25, 2026. support.google.com.
• Google Ads Help. Experiments FAQs. Source of the distinction between budget splits, supported for certain Search, App and Display experiments, and traffic splits, supported for Performance Max, Demand Gen and Video; of the 70-30 allocation example between control and treatment; of the description of the user split occurring at the start of the experiment with balance based on the probability of entering an auction; of the recommendation to run the experiment for at least 4 to 6 weeks; and of the fact that the first 7 days of data is discarded to account for ramp-up time. Checked on September 25, 2026. support.google.com.
• Meta Business Help Center. About daily budgets. Source of the definition of the budget week starting Sunday and ending Saturday, of the statement that Meta may spend up to 75% over the daily budget on days when better advertising opportunities are available, of the ceilings of 175% per day and 7 times the daily budget per week, and of the 210% daily ceiling when ad set budget sharing is turned on. Checked on September 25, 2026. facebook.com.
• Meta Business Help Center. About ad set budget sharing. Source of the definition of sharing up to 20% of the daily budget with other ad sets, of the statement that at a campaign level spend remains the same whether the feature is on or off, and of the stated formulas for maximum spend per day (daily budget plus shared budget, multiplied by 1.75) and per week (the same sum multiplied by 7). Checked on September 25, 2026. facebook.com.

Read also: Ad platform split testing · Smart Bidding · Switchback experiments · Sample ratio mismatch · Conversion lag · Geo experiments · Leia em português