AB Tasty Review 2026: Features, Pricing, Who It Is For
An AB Tasty review for 2026: what the platform does, what can honestly be said about pricing, how to read its win probability, and the segmentation trap.
📚 This article is part of the guide CRO Tools Compared: A Neutral Guide by Category (2026).
AB Tasty treats experimentation and personalization as the same product, and that single design decision explains both its best use case and its main reading trap. The platform covers A/B, split URL, and multivariate testing with a visual editor, audience personalization, feature flags, and a Bayesian statistics engine whose headline indicator is the chance to win. This review covers what it does well, what can honestly be said about price, where the real limits sit, and above all the statistical discipline its emphasis on segmentation demands. For the wider landscape, see our neutral comparison of CRO tools by category.
A disclosure before anything else: Donnu is an A/B testing tool and competes with AB Tasty on part of this scope. This review was written to be useful even to a reader who picks AB Tasty at the end, which is exactly why the section on where it is clearly the right call exists.
What AB Tasty Is, One Line Per Module
| Module | What it does | Who tends to care |
|---|---|---|
| Web experimentation | A/B, split URL, and multivariate testing with a visual editor | Marketing, without waiting on a deploy |
| Personalization | Audience rules and content delivery per segment | Operations with a segmented, high-volume audience |
| Feature flags and rollout | Controlled feature release through SDKs | Engineering and product |
| Recommendations and messaging | Dynamic blocks and on-site communication | Ecommerce and media |
| Bayesian engine | Chance to win and per-variation interval | Whoever reads the result and approves the rollout |
The honest reading of that table: choosing between AB Tasty and a smaller tool is rarely about the quality of the A/B test itself. It is about whether personalization is a real workstream in your operation, with someone accountable for it, or a wish-list item nobody will actually run. Teams with that workstream active find a natural fit, because experiments and personalization share one audience definition. Teams without it are buying a capability they will pay for every month and never use.
One piece of context that matters for any 2026 evaluation: on 20 January 2026, AB Tasty and VWO announced a merger into a single digital experience optimization platform, backed by Everstone Capital, according to the joint announcement from both companies. The official communication describes a combined entity with more than 100 million dollars in annual revenue and more than 4,000 customers, and press coverage of the deal reports Sparsh Gupta leading the combined company. Consolidation at that scale reorganizes packaging, positioning, and roadmap without warning.
Pricing: What Can Honestly Be Stated
Not much, and that is the finding. AB Tasty does not publish figures on its official pricing page: the posture is a custom proposal through a sales conversation. That is the enterprise-band norm in this category rather than a quirk of this vendor, but it carries a real time cost during evaluation and makes side-by-side comparison against publicly priced tools harder.
Four variables set the final number across the whole category:
- Monthly tested visitor volume, the base of nearly every contract.
- Which modules are included, since experimentation, personalization, and feature flags are usually separate line items.
- Number of domains or projects covered.
- Contract length, with discounts for longer commitments.
With the merger in motion, a fifth variable enters the calculation: what happens to your package once the two catalogs are reorganized. That is not a reason to avoid the vendor. It is a reason to prefer shorter terms and to ask, in writing, for the clause that lets you export your experiment data. Our guide to A/B testing tools pricing compared covers how to build that comparison properly.
Chance to Win: What the Bayesian Number Claims, and What It Does Not
The central indicator in the interface is the chance to win. According to AB Tasty official documentation, it is a statistical index expressing the probability of a strictly positive gain of a variation over the original version, shown as a percentage and based on a Bayesian test. The same documentation guides that a chance to win at or above 95 percent means the collected statistics are reliable and the variation can be shipped at what it describes as low risk.
That is a clear, well constructed definition, and it solves a genuine problem: the question a manager actually asks (“what is the chance B is better?”) is literally the question the Bayesian method answers, while a p-value answers something else. The risk is not in the number, it is in the mental translation a team performs on it.
Two misreadings show up constantly:
- “95 percent chance to win is the same as 95 percent confidence.” It is not. Different questions, different math, and the two numbers can disagree on exactly the same data. Our comparison of credible intervals versus confidence intervals works through that difference in detail.
- “A high chance to win means a big win.” Also no. It speaks to the direction of the effect, not its size. A tiny effect measured on an enormous sample produces a very high chance to win and irrelevant business impact. A complete read needs the per-variation interval next to it, never the index alone.
The Discipline Personalization Demands: Segment Reads Inflate False Positives
This is the most important section of this review, and it applies to any tool with a heavy segmentation emphasis, not only to AB Tasty. When a product makes it very easy to slice results by audience, teams naturally slice results by audience. And looking at the same test across several segments multiplies the chance of finding a winner that does not exist.
The Worked Example: A Flat Test That “Wins” On Mobile
A product page test runs to 20,000 visitors per variation. The overall result:
| Read | Visitors per variation | Conversions A | Conversions B | z score | p-value | Verdict |
|---|---|---|---|---|---|---|
| Overall | 20,000 | 800 (4.00%) | 816 (4.08%) | 0.41 | 0.685 | Inconclusive, effect indistinguishable from zero |
Check that row in the calculator:
Two-sided two-proportion z-test. "Not significant" almost always means not enough sample, not that the versions are equal.
Nothing happened. Then someone opens the segment report and looks at six slices: mobile, desktop, paid traffic, organic traffic, new visitor, returning visitor. In the mobile slice, at 3,000 visitors per variation, this appears:
| Read | Visitors per variation | Conversions A | Conversions B | Relative lift | z score | p-value |
|---|---|---|---|---|---|---|
| Mobile segment | 3,000 | 90 (3.00%) | 126 (4.20%) | +40.0% | 2.49 | 0.0126 |
A p-value of 0.0126, well under 0.05, with a 40 percent relative lift. It looks like an excellent finding. It is not, and the reason is plain arithmetic.
Run six independent tests at a 5 percent threshold when no real effect exists anywhere, and the probability that at least one comes back significant by chance is 1 minus 0.95 raised to the sixth power, roughly 26.5 percent. Put differently: in about one out of every four segment analyses like this one, some segment “wins” without anything having actually happened.
The simplest and most conservative correction, Bonferroni, divides the threshold by the number of comparisons: 0.05 divided by 6 is 0.0083. The mobile segment p-value was 0.0126, which is larger than 0.0083. The finding does not survive the correction. It is not a discovery, it is the expected outcome of looking six times.
Why “Just Run the Test On That Segment” Is Not Free
The obvious fix is: if mobile looks promising, run a test only on mobile. It is the right fix, but it carries a cost that has to be on the table before anyone promises a timeline.
Sizing a dedicated mobile test, at a 3% baseline rate and an ambition to detect a 10% relative gain, requires 53,211 visitors per variation. Run the numbers:
Two-proportion normal approximation, 2 variations (50/50). Tweak the inputs and watch it update live.
With the whole flow receiving 21,000 visitors per week, that test takes 36 days. But the mobile segment is only a slice of it: at roughly 3,500 visitors per week in that slice, the same test takes 213 days, nearly seven months occupying that flow.
| Where to run it | Weekly traffic available | Sample per variation | Duration |
|---|---|---|---|
| Whole flow | 21,000 | 53,211 | 36 days |
| Mobile segment only | 3,500 | 53,211 | 213 days |
Sample by normal approximation of two proportions, 3% baseline, 10% relative MDE, 95% confidence, 80% power, two-sided; duration for 2 variations.
That calculation almost never appears in a personalization demo. Segmenting is not free: every slice divides the sample and multiplies the calendar. A platform that makes slicing easy needs to arrive with the discipline not to turn every slice into a decision. Our guide to common A/B testing mistakes covers this family of problems in more depth.
AB Tasty Review: Real Strengths
- Experimentation and personalization on one audience definition. Teams running both workstreams avoid maintaining two segment catalogs in two tools, a classic source of number disagreement.
- A mature visual editor. Years of iteration on an editor that survives real websites, which is not trivial. Marketing can build a variation without waiting on a deploy.
- Explicit, documented statistics. The public documentation defines what chance to win means and how to interpret it, instead of hiding the statistics behind a winner badge.
- Coverage of advanced cases. Multivariate testing, feature flags, recommendations, and on-site messaging exist in the portfolio, so a maturing program does not have to switch tools.
Limitations and Watch-Outs
- Opaque pricing by design. With no published figures, evaluation requires a sales conversation, a real time cost that also makes direct comparison harder.
- Segmentation invites false positives. Not a product defect, a consequence of what the product makes easy: without an internal rule on how many segments may be inspected and what happens to a finding, the team decides on noise regularly.
- Complexity proportional to breadth. A platform spanning experimentation and personalization needs someone who owns it. Without a clear owner, the personalization half usually sits idle.
- Contracting in a consolidating market. With the merger announced in January 2026, long terms deserve more scrutiny than usual, and keeping experiment data exportable becomes a practical requirement.
How to Evaluate AB Tasty in a Pilot
| Check | How to do it | What it reveals |
|---|---|---|
| Real traffic split | Run an A/A test for a few days and verify the proportion | Whether the split is stable and free of sample ratio mismatch |
| Segment discipline | Define upfront which segments will be inspected, in writing | Whether the team can avoid deciding on noise |
| Statistical reading | Compare chance to win against a frequentist read of the same data | Whether the team understands the two answer different questions |
| Performance impact | Measure the page with and without the script, same device and network | What the test costs in performance score |
| Export | Ask for raw experiment data in an open format | Whether you can leave, and with what in hand |
The second row is the one that matters most with this specific tool, and it is a process check, not a product check. Write down before the test which segments will be analyzed and what happens if one of them wins alone. The honest answer is almost always “it becomes a hypothesis for a dedicated test”, never “ship it”.
The A/A test in the first row deserves its own note: both variations are identical, so any “winner” that appears is noise by definition. It is the cheapest audit of a new tool that exists, and it is worth running on any tool you are evaluating, including ours.
Who AB Tasty Is For
It fits mid-market and large operations where personalization by audience is a real workstream with a defined owner and enough traffic to segment without losing statistical power, and teams that want experiments and personalization sharing one audience base.
It fits less well for a small operation that only needs A/B testing on web pages, for buyers who want public pricing and self-service without a call, and for teams whose per-segment traffic cannot yet support a segmented read, a scenario where it is worth returning to the fundamentals in our complete conversion rate optimization guide before buying segmentation.
If your case is the second one, Donnu is one of the lighter options in the category, with predictable pricing and no call required to learn the number, focused on web A/B testing with honest statistics. It does not replace AB Tasty for audience personalization, recommendations, on-site messaging, or SDK feature flags, and claiming otherwise would be dishonest: if you need those pieces, Donnu is not the right comparison. If you do not, start a free trial and compare what actually matters for your case.
Read also: VWO review 2026 · AB Tasty vs Kameleoon · CRO tools compared · Common A/B testing mistakes · Leia em português
References
- AB Tasty. Statistics for the reporting. Official documentation on the statistics engine and report indicators. docs.abtasty.com.
- AB Tasty. Statistical metrics. Official definition of the chance to win index and the guidance to read it from 95 percent. docs.abtasty.com.
- AB Tasty and VWO. VWO and AB Tasty Join Forces to Redefine the Future of Digital Experience Optimization. Joint merger announcement, 20 January 2026. abtasty.com/news/vwo-ab-tasty-join-forces.
- GlobeNewswire. VWO and AB Tasty Join Forces to Redefine the Future of Digital Experience Optimization. Official release distributed on 20 January 2026, with the combined entity figures. globenewswire.com.
- Kohavi, R., Tang, D. and Xu, Y. Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press, 2020. Chapters on segment analysis and multiple comparisons. Companion material at experimentguide.com.
- Gelman, A. et al. Bayesian Data Analysis, 3rd edition. Reference for the Beta-Binomial model and reading posterior probabilities. sites.stat.columbia.edu/gelman/book.
Frequently asked questions
- How much does AB Tasty cost in 2026?
- AB Tasty does not publish a price list on its official pricing page: the stated posture is a custom quote per account, which is the norm for enterprise-tier platforms in this category. In practice the number depends on monthly tested traffic, which modules you take (experimentation, personalization, feature flags), how many domains are covered, and contract length. Ask for the quote broken down by traffic band and by module, and get in writing what happens if your traffic overshoots the band mid-contract.
- What is AB Tasty chance to win, and how should you read it?
- According to AB Tasty official documentation, the chance to win is a statistical index expressing the probability that a variation produces a strictly positive gain over the original, shown as a percentage and computed with a Bayesian test. The same documentation suggests that a chance to win at or above 95 percent indicates low risk in shipping the variation. It is a legitimate, well defined number, but it answers a different question than a p-value does, so the two are not interchangeable.
- Is a 95 percent chance to win the same as 95 percent statistical confidence?
- No. Chance to win answers "given what I observed, how likely is it that the variation is better than control". A frequentist confidence interval answers a question about how often the procedure would be right across hypothetical repetitions of the experiment. The two numbers can disagree on exactly the same data, and confusing one for the other is the single most common reading error among teams moving between tools with different statistical engines.
- Does the VWO merger change anything for AB Tasty customers?
- On 20 January 2026, AB Tasty and VWO announced they were merging into a single digital experience optimization platform, backed by Everstone Capital, according to the joint announcement from both companies. The official communication describes a combined entity with more than 100 million dollars in annual revenue and more than 4,000 customers. For an existing customer the immediate effect tends to be small, but consolidation of that size reorganizes packaging, positioning, and roadmap over time, which is an argument for shorter contract terms and for keeping your experiment data exportable.
- Can you trust a result that only shows up in one segment of a test?
- Very carefully, and almost never as a standalone decision. Look at the same flat test across six segments and the probability that at least one crosses the 5 percent threshold by chance alone rises to roughly 26.5 percent, even when no real difference exists anywhere. The honest way to handle a segment finding is to log it as a new hypothesis and run a dedicated test sized for that segment, instead of shipping it directly.
- Who is AB Tasty a good fit for, and who should look elsewhere?
- It fits mid-market and large operations where personalization by audience is a real, staffed workstream, with enough traffic to segment without losing statistical power, and teams that want experimentation and personalization sharing one audience definition. It fits less well for a small operation that only needs A/B testing on web pages, for buyers who want public pricing and end-to-end self-service, and for teams whose per-segment traffic cannot yet support a segmented read.