Freemium Paywall Experiments: What to Test and How to Decide
Freemium paywall testing: where to place the trigger, which metric actually decides the test, and how to size a test when free-to-paid is low.
📚 This article is part of the guide How to A/B Test Pricing: The Complete Guide.
A/B testing where and how a paywall shows up inside a freemium flow is one of the highest-leverage experiments in the complete guide to A/B testing pricing, and also one of the easiest to misread. A more aggressive paywall almost always converts more in the short run, and the question that decides whether that was a genuine win or a problem in disguise is what happens to the retention of the people it converted under pressure. This article covers where to position the trigger, what is actually worth testing, the metric that should decide the test, and how to size the sample when free-to-paid conversion is low, which describes most freemium products.
Where to Place the Paywall Trigger in the User Journey
Every freemium product has a decision point built into it somewhere: the user hits a usage cap, a locked feature, or a message inviting them to upgrade. That trigger can be a usage limit (number of projects, rows of data, exports per month), a feature that only exists on the paid tier (team collaboration, an integration, a report), or simply elapsed time (the end of a trial window). The most common mistake is choosing that trigger based on what is easiest to build, rather than on the moment the user has already felt the product’s core value, what growth teams usually call the “aha moment.”
RevenueCat, analyzing paywall placement across subscription apps, recommends testing trigger position along several points in the funnel: before onboarding, after onboarding, before a locked feature is used, and after it is used. In RevenueCat’s own case studies, apps that moved the paywall earlier, closer to or inside onboarding, saw large jumps in top-of-funnel conversion: the plant-care app Greg reported trial signups rising roughly 400% and sign-up-to-trial conversion climbing from 3% to 15% after moving its paywall into onboarding, and the wellness app Rootd reported a 5x revenue increase from a similar move. Those case studies are about immediate conversion, not about what happens to those subscribers afterward, which is exactly the gap this article is built around. That is the central point of this article: which timing wins for the metric you actually care about, net of retention, is a decision only your own A/B test can make for your specific product, because the answer shifts with how quickly a given user reaches the core value.
What to A/B Test in a Freemium Paywall
Once the general trigger category is set, three variables inside it are worth testing on their own:
- Trigger position. Moving the limit, whether usage-based or feature-based, earlier or later in the flow. This is the highest-impact variable and also the riskiest, because it touches conversion and retention at the same time, sometimes in opposite directions.
- Message framing. The copy and framing at the paywall itself: loss framing (what the user gives up) versus gain framing (what they get), urgency (a counting-down trial versus no deadline), and how clearly the paid plan’s value is stated at that exact moment of being blocked.
- How much runway the free tier gives before charging. How many actions, exports, or days of use the free plan allows before the block appears. More runway tends to raise the odds a user feels the value first, but it delays monetization and risks never converting someone already satisfied with the free tier.
The Metric That Decides the Test (and the Mistake Most Teams Make)
Here is the central point of this article: measuring only the click on “upgrade,” or free-to-paid conversion at the moment of the test, is measuring the wrong half of the story. An earlier or more aggressive paywall almost always raises that immediate conversion number, simply because it interrupts more people, at more moments, with more insistence. What that metric alone cannot show you is how many of those new subscribers actually wanted the product, versus how many subscribed under pressure and will cancel at the first billing cycle once they realize they do not use it enough to justify the price.
The correct primary metric is a pair, and both halves need to be read together:
- Free-to-paid conversion, measured over the window that fits your sales cycle (usually counted from signup, not from the moment of the block, so you do not discard users who never reached the paywall at all).
- Post-conversion retention, over a fixed window (30 days is a common starting point): of the people who converted, how many are still paying customers once that window closes.
A paywall variant that wins on metric one and loses on metric two is not a win, it is one problem traded for another, deferred until after the test has already been called.
A Worked Example With Real Numbers
Say your product currently converts 3.0% of signups from free to paid. You want to test moving the trigger earlier (variant B) against the current setup (control A), with 8,000 signups analyzed on each side.
Free-to-paid: A converted 240 of 8,000 (3.0%); B converted 296 of 8,000 (3.7%). Running those numbers through a two-proportion z-test gives a relative lift of +23.3%, z is approximately 2.46, the p-value is approximately 0.0139, which is significant, with B winning. Read on its own, this metric says “move the paywall earlier.”
Now look at 30-day retention among the people who converted: in A, 180 of the 240 subscribers (75.0%) were still paying 30 days later; in B, 189 of the 296 subscribers (63.9%) were. That difference is also significant: a relative lift of -14.9%, p-value approximately 0.0056, with A winning on this metric.
So which variant “won”? That depends on what you do with the information, not on which metric you choose to ignore. Multiply conversion by retention for a combined read: A retains 240 x 75% = 180 active subscribers at day 30; B retains 296 x 63.9% is approximately 189 active subscribers at day 30. In this specific scenario, B still edges ahead in raw retained subscribers, but the gap is much smaller than the +23.3% headline conversion lift suggests on its own, and if your paid plan carries a higher support cost per unstable customer (more tickets, more refunds, more cancellations that land outside the 30-day window), the calculation could flip entirely. The point is not that a single correct answer exists in advance: it is that you only get to it by measuring both metrics together, not by picking the one that looked good first.
Sizing a Test When the Free-to-Paid Base Rate Is Low
Small baseline conversion rates require disproportionately larger samples to detect the same relative effect, which is simply how any A/B test’s math works, except freemium products tend to sit right in the most expensive part of that curve. According to the SaaS Conversion Report by Kyle Poyar (Growth Unhinged), in partnership with ChartMogul and ProductLed, based on data from roughly 200 B2B software products, a “good” free-to-paid conversion rate for a self-serve freemium product falls between 3% and 5%, “great” sits between 8% and 12%, and a full quarter of products convert below 2.5%. Plug your own real conversion rate and the effect size you want to detect into the calculator below before running any paywall test:
Two-proportion normal approximation, 2 variations (50/50). Tweak the inputs and watch it update live.
With a 3.0% baseline and a minimum detectable effect of 15% relative (the difference that would actually be worth knowing about), the required sample is 24,193 signups per variant. At 4,000 new signups per week, that means running the test for roughly 85 days, nearly three months. That is considerably longer than most teams budget for a paywall test, which is exactly why freemium monetization experiments tend to get called too early or read off an underpowered sample.
Common Mistakes in Freemium Paywall Testing
| Mistake | Why it happens | Fix |
|---|---|---|
| Moving the paywall earlier than the aha moment | It looks like a fast way to lift short-term revenue | Test with retention as a joint metric, never conversion alone |
| Measuring only the short-term metric | The conversion dashboard updates fast; retention takes weeks to show up | Set the retention window (e.g. 30 days) before the test starts, and declare a winner only after it closes for both sides |
| Sizing the test by gut feel | A low free-to-paid rate makes teams assume “any volume is fine” | Calculate the sample from your actual rate with the calculator above; low rates need more traffic, not less |
| Ignoring the support cost of unstable subscribers | The gross revenue lift looks good in isolation | Where possible, fold refunds and support tickets into a net revenue read before declaring a winner |
Paywall Placement by Product Type
Where the trigger tends to work best shifts with how the product delivers value, because the “aha moment” lands at a different point for each shape:
| Product type | Where the trigger tends to work best | Why |
|---|---|---|
| Productivity tools (notes, spreadsheets, task management) | A limit on items or active projects, not on time in use | Value is felt by accumulating your own content; blocking by elapsed time punishes users who have not migrated their workflow yet |
| Team / collaboration and messaging | Message or integration volume, after a generous initial window | Whole-team retention depends on the product taking hold across the group before any charge appears, a slower process than for a single user |
| Ecommerce / store tooling | An advanced feature (reporting, automation, an extra channel) after the first recorded sale | The aha moment is the first sale attributed to the tool; charging before that interrupts the proof of value at the worst possible point |
Treat this table as a starting hypothesis, not a verdict: the point of this article is that placement gets tested, not copied from a table.
Write the Hypothesis Before You Move the Trigger
Like any other A/B test, a paywall test without a written hypothesis invites “discovering” a winner by accident among whichever secondary metric happened to move (clicks on the upgrade banner, time to first block, paywall dismissal rate). Write it down before the test runs: what evidence motivated the change in trigger position, what effect you expect on each of the two primary metrics, and which retention window will decide the test. That avoids the subtler trap in this kind of experiment: celebrating the number that moved fast, and only discovering its real cost weeks later, after the test has already been closed and the learning lost.
It is also worth setting a support guardrail: if the support team reports a jump in cancellation or refund tickets during the test, that is a warning sign independent of the statistical significance on the two primary metrics, and it is worth pausing the test to investigate before letting it run to completion.
Make This Automatic on Donnu
The real work of a paywall test is not deciding “earlier or later,” it is balancing short-term conversion against real retention, with significance validated on both metrics instead of celebrating whichever number moves first. Donnu A/B runs the same two-proportion math on both ends of your monetization funnel, so you can see whether a more aggressive paywall is actually paying off or just deferring a cancellation to next month, without building that cross-metric math by hand.
Start a 14-day free trial and test your next paywall trigger by measuring conversion and retention together, not one in place of the other. See also the complete guide to A/B testing pricing and price anchoring experiments, the sibling pricing experiment.
Read also: How to A/B Test Pricing: The Complete Guide · Price Anchoring Experiments · Sample Size Calculator
References
- Kyle Poyar (Growth Unhinged), ChartMogul, and ProductLed. The SaaS Conversion Report. chartmogul.com/reports/saas-conversion-report.
- RevenueCat. Optimizing paywall placement: The key to unlocking more app subscribers. revenuecat.com/blog/growth/paywall-placement.
Frequently asked questions
- Where should I place the paywall in a freemium product?
- After the user has already experienced your product's core value at least once, never before. Placing the paywall before that "aha moment" is the most common way to quietly kill long-term conversion: the user has no idea what they would be giving up, so saying no costs them nothing. The right trigger varies by product (a usage limit, an advanced feature, team collaboration), but the right timing is consistently after value, not before it.
- Which metric should decide a freemium paywall A/B test?
- Two metrics read together, never one alone: free-to-paid conversion, and retention among converted users over a fixed window (30 days is a common starting point). A more aggressive paywall almost always lifts immediate conversion, but it can also raise early cancellations, because it converts people who had not yet decided the product was worth paying for. Measuring only the click on "upgrade" hides that second effect completely.
- How do I size a paywall test when free-to-paid conversion is low?
- The lower your baseline conversion rate, the larger the sample you need to detect the same relative effect, and freemium products typically convert in the low single digits. Plug your actual free-to-paid rate into the sample size calculator on this page, not a generic "industry average," and budget for the test to run considerably longer than a typical page-level experiment.
- Does moving the paywall earlier always increase revenue?
- Not necessarily, which is exactly why it needs to be tested rather than decided by intuition. Moving the trigger earlier tends to increase immediate upgrade clicks, but if users have not yet felt the product's value, more of them cancel in the first billing cycle, and net revenue after churn can end up worse than a later paywall would have produced, even though the initial conversion rate looked stronger.
- What is the difference between a hard paywall and a metered paywall for freemium products?
- A hard paywall blocks the entire product until payment, common in trial-based apps; a metered paywall lets users keep a permanently free tier with a usage cap or feature limit, which is the more common shape in self-serve SaaS freemium. The trade-off in this article, immediate conversion versus post-conversion retention, applies to both shapes, but a metered paywall gives you more test surface: you can move the meter itself, not just the message shown when it triggers.