Mobile Paywall A/B Testing: A Practical Guide
Mobile paywall A/B test: where to place the trigger, hard vs soft paywall, the metric that decides it, and the statistics iOS and Android demand.
📚 This article is part of the guide Mobile App A/B Testing: The Complete Guide.
A mobile paywall A/B test compares two versions of how and when you ask the user to pay, measuring which one produces more subscription conversion without destroying retention afterwards. It is the single test that decides most of the revenue of a subscription app, and it is also the easiest one to misread, because a more aggressive paywall almost always lifts first-moment conversion even when it damages the business in aggregate. This article is a child of the complete guide to A/B testing in mobile apps and assumes the same statistical base explained there: device-level randomization, remote feature flags instead of instant deploys, and extra care with SRM between app versions.
If your focus is still the first experience before the user ever reaches the paywall, read the mobile onboarding A/B testing guide as well, because where the paywall sits inside onboarding is in practice a joint decision between the two experiments.
Where to place the paywall
Paywall timing is the single variable that moves a mobile monetization test the most, more than copy and more than visual design. Three moments account for most real experiments.
- Before onboarding (immediate hard paywall). The user sees the subscription offer before using the product at all, sometimes with a fixed-length trial attached. It converts better among people who arrived already decided to pay (qualified paid traffic, for example), and it pushes away part of the audience that only wanted to try, shrinking the volume that ever activates.
- After the aha moment (mid-funnel paywall). The user has already felt the core value (completed the first relevant action, seen the first result) before any offer appears. It tends to convert a smaller but more convinced base, because the decision to pay follows a real experience instead of a promise.
- At a usage limit (soft paywall). The user runs a genuinely free version until hitting a ceiling (a number of searches, a cap on saved items, a specific premium feature), and only then sees the offer. It is the most common model in productivity and tool apps, because it lets the product prove value before charging.
Hard vs soft paywall: what each one is really testing
Hard and soft paywalls are not just different positions in the funnel, they are different business models, and testing one against the other is a bigger decision than testing copy or button color.
| Criterion | Hard paywall | Soft paywall (freemium) |
|---|---|---|
| Access to the product without paying | None, or only a fixed-length trial | Real, up to a usage or feature limit |
| Download to paid conversion by day 35 (median, RevenueCat) | 10.7% | 2.1% |
| Top 10% of apps, same download to paid cut (RevenueCat) | 38.7% | not measured in the same cut by RevenueCat |
| Installs that reach the value moment | Lower (non-converters leave early) | Higher (a real free product keeps more people) |
| What you typically test | Trial length, prompt timing, urgency message | Where the limit sits, what is free, the offer trigger |
| Main risk | Repelling users who only wanted to try | Never converting people who already get value for free |
According to the same RevenueCat report, trial length moves the outcome sharply even inside the hard paywall model: trials of 17 to 32 days convert at a median of 42.5%, against 25.5% for trials shorter than 4 days, and 55% of the cancellations of 3-day trials happen on day 0, before the user has experienced the product for a full day. That does not tell you which length is right for your product, but it does show that “how long should the trial be” is itself a test variable as consequential as paywall copy.
What to test on a mobile paywall
Four variables account for most paywall tests with a measurable effect.
- The message. What the paywall promises (time saved, one specific feature, social proof) changes perceived value before price is even read. Test one concrete benefit against a generic feature list.
- Trigger position. When the paywall appears (by time in product, by attempting a gated feature, by session count) changes who sees it and in what state of intent.
- Hard vs soft paywall. As described above, the highest-impact and highest-risk test, because it changes the acquisition model, not just a screen.
- Plan presentation: annual vs monthly. Which plan is preselected, how the annual plan is anchored (equivalent monthly price, percentage discount, absolute savings) and the display order affect both conversion and which plan is chosen, which moves lifetime value even when gross conversion is flat.
On that last point: testing which plan comes preselected usually has an effect out of proportion to the implementation effort, because most users accept the option already marked instead of actively switching. It is also where Apple is strictest. App Store Review Guideline 3.1.2 requires the total amount charged to be the most prominent and legible pricing element on the purchase screen, with any promotional price, trial or discount calculation displayed in a subordinate, smaller position. In practice that rules out variations designed to hide or downplay the full price in order to push the annual plan, the same kind of variation that would sail through on a web paywall.
The metric that decides: free to paid conversion plus post-conversion retention
The principle matches any paywall or pricing test on the web: the primary metric is free (or trial) to paid conversion, always read alongside a post-conversion retention guardrail, because converting more people who churn quickly is not a win, it just moves the loss forward. The mobile nuance sits in two places that barely exist in a web checkout.
- Cancelling is easier inside the app store. Apple requires subscription cancellation to happen in the subscription manager of the user’s own Apple account; the developer’s app has no way to process that cancellation directly. Google also guarantees cancellation through the Play Store, and its subscription policy now requires the developer’s own app to offer an easy cancellation path, in at most two taps, banning design patterns that hide or obstruct that option. In both cases the developer does not control how hard cancelling can be made, unlike a web checkout where the product owns the flow (and sometimes obstructs it deliberately). The practical consequence: an aggressive paywall shows its churn effect faster on mobile than on the web.
- Store refund windows are outside your control. According to Google Play support, subscriptions are refundable within 48 hours of the initial purchase through the standard request process; after that, cancelling stops future charges but does not return the period already paid. Apple, per its own official support page, recommends requesting a refund within 14 days but technically accepts requests up to 90 days after purchase, reviewing each case individually. Neither window is defined by your product, so any net revenue calculation from a paywall test has to work with the store’s window, not the refund window you would use in your own checkout.
Treat both as guardrails rather than footnotes: a paywall that wins on gross conversion while also lifting refunds and first-month cancellations may simply be compressing in time a revenue stream that was leaving anyway.
Mobile specifics that change the test
Three mobile-specific factors threaten the reading of a paywall test in ways that would go unnoticed on a website test.
SRM between app versions
If the paywall variation depends on a new app version or on a remote configuration that has not reached everyone, a slice of the base falls into control by default (or never enters the test), with nobody deciding it. A worked example: a paywall test configured 50/50 should expose 5,000 users to each variation, out of 10,000 who reached the paywall moment in a week. Because the new variation only renders on app versions at or above the update, the observed result was 6,200 users in control and 3,800 in the new variation. The goodness-of-fit chi-square for that divergence ((6200-5000)² / 5000 + (3800-5000)² / 5000) comes to 576, far above the 1% critical threshold of 6.63 with 1 degree of freedom, meaning a split of that magnitude is effectively impossible by chance. The likely cause is technical (app version or remote config cache), not statistical, and no conversion result from that test should be trusted until the cause is fixed.
Remote config propagation latency
Even when the paywall variation does not require a new binary (only a flag read through remote config), the device still has to fetch that configuration before rendering it, and the process is not instant. An app backgrounded for days, or running on a device with aggressive battery saving, can keep showing the old paywall for far longer than expected. Treat the first days of any paywall test as a propagation period, not as valid sample to be added to the final reading without checking.
Store rules about price on the paywall
Beyond Apple Guideline 3.1.2 already cited, Google Play natively offers a Price experiments tool in the Play Console, built to test price with real traffic; Google itself recommends waiting for statistical significance before applying the winning variation, and that application is manual (a button the developer presses), not automatic. Today the tool is limited to one-off in-app products, not the subscription price behind the paywall. Apple has no equivalent native price testing tool inside App Store Connect; testing price, copy and paywall layout on iOS depends on your own remote config mechanism, within the limits Guideline 3.1.2 sets on the prominence of the amount charged.
A worked example with real numbers
Scenario: an app with a soft paywall (free use up to a limit, then the offer) converts 7% of the users who see the paywall from free to paid. The team redesigns the message and the trigger moment, expecting a 20% relative improvement (from 7% to about 8.4%). At 95% confidence and 80% power, the same defaults used in every calculator on this blog, adjust the numbers below for your own scenario:
Two-proportion normal approximation, 2 variations (50/50). Tweak the inputs and watch it update live.
Running that calculation through the same math the calculator uses (sampleSizePerVariant), the result is 5,691 users per variation (11,382 in total, counting people who actually reach the paywall, not total installs). If 1,800 users per week reach the paywall moment, a realistic volume for a mid-sized app on a conversion screen and far below total installs, the test needs 45 days, about six and a half weeks, for both variations to fill the sample.
Now look at what happens if the team peeks early, with only a third of the calculated sample (1,897 per variation) and observed conversion rates almost identical to the full scenario:
| Full sample (5,691 per variation) | Partial sample, one third (1,897 per variation) | |
|---|---|---|
| Rate A (control) | 6.99% | 7.01% |
| Rate B (variation) | 8.40% | 8.38% |
| Observed relative lift | +20.1% | +19.5% |
| p-value | 0.0049 | 0.113 |
| Confidence interval of the difference | +0.43 to +2.38 points | −0.33 to +3.07 points |
| Verdict | Significant, B wins | Inconclusive |
The observed improvement is practically the same in both cases (the new variation really does look about 20% better), but the verdict flips completely. With the full sample the p-value lands at 0.0049 and the confidence interval of the difference does not cross zero, sitting between 0.43 and 2.38 percentage points, a solid result. With a third of the sample the p-value climbs to 0.113, above the 0.05 threshold, and the interval crosses zero (from −0.33 to +3.07 points), meaning it is still plausible that the new paywall makes no difference at all. That does not mean the partial test “failed”, it means there was not enough sample for the same real effect to become statistical evidence, and it is exactly why the calculated 45-day duration exists: so nobody decides by staring at the dashboard halfway through.
Common mistakes in mobile paywall tests
| Mistake | Why it happens | Consequence |
|---|---|---|
| Measuring conversion only, with no retention guardrail | First-moment conversion is easier to see than churn weeks later | A “winning” paywall that only accelerated a revenue loss that was already coming |
| Not segmenting SRM by app version | The new variation only reaches users who already updated, and it hides in the aggregate | Invalid conversion result, even when it looks clean at first glance |
| Confusing an ASO test with a paywall test | Both compare “two versions of something”, at different points of the funnel | Optimizing icon and screenshots does not move in-app subscription conversion, and vice versa |
| Ignoring remote config propagation time | The flag is read on the next app sync, not instantly | The first days of the test are contaminated by users still seeing the old version |
| Testing price variations that hide the amount charged on iOS | Rushing to replicate a test that would work on a web paywall | Risk of rejection in Apple review for violating Guideline 3.1.2 |
| Declaring a winner on a partial sample | Anxiety to ship, checking the dashboard too early | The same real effect can read as inconclusive purely for lack of sample, as in the worked example above |
ASO and paywall: two different things
The two get confused because both “test two versions of something app related”. An ASO (App Store Optimization) test runs on the store listing, before installation: Apple offers it as Product Page Optimization in App Store Connect, testing visual elements only (icon, screenshots, preview video), and Google offers Store listing experiments in the Play Console, which goes beyond visuals and also tests text (short and full description), both with real store search and browse traffic. What decides an ASO test is listing view to install conversion. An in-app paywall test runs afterwards, inside the installed product, and what decides it is free (or trial) to paid conversion. An app can win an ASO test and still carry the same weak paywall, or the reverse: they are independent optimizations on different metrics, both worth running, and neither is a substitute for the other. The app store listing A/B testing guide covers the store side in detail.
Make this automatic with Donnu
This guide covered what genuinely changes when a paywall test moves out of a web checkout and into the app: hard vs soft paywall as a business model decision, SRM that usually starts with an outdated app version, remote config propagation latency, and store rules on the prominence of the amount charged, all on top of the same sample and significance discipline any A/B test demands.
Donnu today is a tool focused on the web and client-side layer: a light snippet that never blocks the page, automatic sample sizing and honest Bayesian statistics. It does not test paywalls inside a native iOS or Android app, and this guide does not claim otherwise. If your product has a web layer that takes part in the payment journey (a landing page selling the plan before installation, a complementary web checkout, a logged-in area also reachable from a browser), Donnu applies that same statistical rigor to that slice with a 14-day free trial.
Read also: A/B Testing in Mobile Apps: The Complete Guide · Mobile Onboarding A/B Testing · Leia em português
References
- RevenueCat. State of Subscription Apps 2026. Trial and free to paid conversion benchmarks by paywall model and trial length, based on more than 100,000 apps. revenuecat.com/state-of-subscription-apps.
- Google Play Console. Price experiments. Official page on the native price testing tool for one-off in-app products, with manual application of the winning variation recommended after statistical significance; it does not cover subscription price today. play.google.com/console/about/price-experiments.
- Apple Developer. App Review Guidelines (section 3.1.2, Subscriptions). Official rules on the prominence of the amount charged and subscription information on the purchase screen. developer.apple.com/app-store/review/guidelines.
- Apple Support. Request a refund for apps or content that you bought from Apple. Official refund request window and process for purchases and subscriptions. support.apple.com/en-us/118223.
- Google Play Help. Apps, games, and in-app purchases (including subscriptions) refund policies. The 48-hour window and refund request process for Google Play subscriptions. support.google.com/googleplay/answer/15574908.
- Google Play Console. Store listing experiments. Native A/B testing of the store listing (icon, feature graphic, screenshots, video and also short and full description), distinct from an in-app paywall test. play.google.com/console/about/store-listing-experiments.
- Apple. Product Page Optimization. The App Store Connect feature that tests icon, screenshot and preview video variations of the listing, also distinct from the in-app paywall. developer.apple.com/app-store/product-page-optimization.
Frequently asked questions
- When is the best moment to show a paywall in a mobile app?
- There is no universal moment, which is exactly why this is a test and not a design preference. Three positions account for most real experiments: before onboarding (an immediate hard paywall), right after the aha moment (once the user has felt the core value), and at a usage limit (a soft paywall, where the product is genuinely free up to a ceiling). Each one converts differently depending on how fast your product delivers perceptible value, and you should judge all three by real free to paid conversion, never by taps on the subscribe button.
- What is the difference between a hard paywall and a soft paywall?
- A hard paywall blocks access to the product until the user subscribes, or at most opens a fixed-length trial before billing. A soft paywall lets the user run a genuinely free version, limited by usage or by feature, and only surfaces the paid offer when that limit appears. According to RevenueCat, in its State of Subscription Apps report, apps with a hard paywall convert at a median of 10.7% from download to paid by day 35, against 2.1% for freemium (soft paywall) apps, roughly a fivefold gap. Note the denominator: that figure counts every download, not only the users who started a trial. That does not decide the model for you: a hard paywall also shrinks the number of installs that ever experience the product at all.
- Is free to paid conversion the only metric in a paywall test?
- No, and treating it as the only metric is the most expensive mistake in this category of test. Free to paid conversion decides the test, but it is always read alongside a post-conversion retention guardrail (does the subscriber keep paying in the following months) and, on mobile specifically, alongside store refund rate, because asking for a refund inside an app store is far easier for the user than in any web checkout. An aggressive paywall can inflate day-one conversion and depress real revenue weeks later.
- What is SRM between app versions, and why does it threaten a paywall test?
- SRM (Sample Ratio Mismatch) is when the observed split between variations diverges from the configured split because of a collection problem, not chance. In paywall tests the most common cause is a new variation that depends on an app version or a remote configuration that has not reached part of the base yet: those users fall back into control, inflating one side with nobody deciding it. Always segment your SRM check by app version before trusting any conversion number from a paywall test.
- Is an in-app paywall test the same thing as an ASO test?
- No, they are different mechanisms at different points of the funnel. A paywall test runs inside the installed app, after the user opens the product, and measures subscription conversion. An ASO test, such as Apple Product Page Optimization or Google Play Store listing experiments, runs on the store listing before installation and measures view to install conversion. Apple tests visual elements only (icon, screenshots, preview video); Google Play goes further and also tests text (short and full description). Neither of them tests price or the subscription flow inside the product.
- Can I test mobile subscription pricing the same way I test paywall copy?
- Technically yes, with more regulatory caution than a copy or trigger-position test. The Google Play Console has a native tool called Price experiments, but today it covers one-off in-app products rather than subscription price itself; for subscriptions Google offers other mechanisms, such as price increases with advance notice to existing subscribers. On the Apple side, App Store Review Guideline 3.1.2 requires the amount charged to be the most prominent and legible pricing element on screen, with any trial or promotional text subordinate to it, which limits which price layout variations are safe to test on an iOS paywall.