What Is a Contextual Bandit? (vs. A/B Testing)

A contextual bandit answers a question that classic A/B testing, and even a plain multi-armed bandit, never asks: what if the best variation is not the same for everyone? It is a decision algorithm that reads features about each visitor (device, location, referral source, past behavior) and uses that context to choose the action most likely to convert for that specific person, rather than searching for one variation that wins on average across your whole audience. This guide is a deep dive that builds on our companion piece, Multi-Armed Bandits vs. A/B Testing: here we cover the contextual version specifically, the one that turns a bandit into something closer to an automated personalization engine.

Plain Bandit vs. Contextual Bandit: The Difference That Matters

A plain multi-armed bandit, sometimes called a context-free bandit, treats every visitor as interchangeable. It tries each arm (each variation), tracks which one converts best on average, and gradually shifts traffic toward the apparent winner, the mechanic covered in our bandits guide. The question it answers never changes: “which single arm is best, on average, for everyone who lands on this page?”

A contextual bandit changes the question itself. Before picking an arm, it looks at a set of features describing whoever just arrived, the “context”: device type, geography, referral channel, time of day, past-visit behavior, and sometimes richer signals like the product category a visitor browsed last. With that context as an input, the model can learn that arm A converts better on mobile while arm C converts better for returning customers, and serve each of them to the right audience, at the same time, inside the same experiment. There is no longer a single overall winner. There is a winner per slice of context.

It helps to place this against what most readers already know: A/B testing measures the average of two versions. A plain multi-armed bandit chases that average faster, shifting traffic while it is still collecting data. A contextual bandit swaps the question “which average is better” for “what is better for this person, right now.” It is, in essence, automated personalization built on the bandit framework.

How the Model Learns: From Context to Policy, Without the Heavy Math

You do not need to understand the optimization math to use a contextual bandit responsibly, but it helps to understand its central piece: the policy. A policy is the learned rule that maps a context to an action. It is not hand-written like a rule engine (“if device is mobile, show X”); it is estimated from exploration data, adjusting weights that connect each context feature to the expected reward of each arm.

The loop works like this. A visitor arrives and the system captures the available context (device, source, past behavior, and whatever else is instrumented). The current policy uses that context to estimate, for each arm, the expected reward, then picks an action, almost always with a deliberate dose of exploration (trying arms it is not yet fully sure about, instead of always defaulting to the current apparent favorite). The system observes the real outcome (converted or not) and uses that context-action-reward triple to update the policy, sharpening the estimate for the next similar visit.

Two ideas keep this loop from turning into a shot in the dark. The first is the balance between exploration and exploitation: the model has to keep testing uncertain arms in certain contexts, even when another arm looks like the winner there, or it will never find out whether that early impression was just noise. The second is that the policy’s quality depends directly on the quality and volume of context data: a context that is too thin (just “mobile or desktop”) learns little; a context that is too rich, with too little data to support each combination, learns slowly and noisily. Finding that middle ground is most of the operational work of running a contextual bandit.

Real Examples of Contextual Bandits in Production

The idea gets more concrete with use cases already running in production today.

• Content recommendation. Streaming services and news platforms use contextual bandits to decide which headline, thumbnail, or next video to show, with the context being the session’s own consumption history and profile. It is the textbook case cited throughout recommender-systems literature.
• Segment-level dynamic pricing. Instead of testing one single price against another for everyone, a contextual bandit can learn that a specific discount converts better for one traffic segment (say, price-sensitive paid-channel visitors) while a different offer works better for recurring organic traffic, adjusting the offer by context instead of by a fixed rule.
• Banner and CTA personalization by visitor type. A site can hold three top-banner variations and use context (new vs. returning visitor, mobile vs. desktop, traffic source) to decide which one to show to each type of person, instead of picking one single winner for the entire page.
• Coupon and discount allocation. A variant of the pricing case: deciding who receives which coupon, and at what value, based on purchase-propensity signals, is a documented application of contextual bandits in e-commerce today.

The thread connecting all of these cases is the same: there is more than one correct decision available, and the context on hand is rich enough to tell them apart.

Contextual Bandit vs. Rule-Based Personalization

The comparison that matters most for a team already doing some personalization is not with A/B testing, it is with manual, rule-based personalization (“if the visitor comes from Instagram and is on mobile, show banner X”).

Criterion	Manual rules	Contextual bandit
Who decides the context-to-action map	A person, from intuition or ad hoc analysis	A model, from observed real outcomes
Scale of combinations	Workable up to a few dozen rules, then hard to maintain	Handles many context combinations without anyone hand-writing each one
Speed of adaptation	Only changes when someone edits the rule	Adjusts continuously as new data arrives
Explainability	High: you can point to the exact rule and the reason	Lower: the decision is a combination of statistical weights, not a single sentence
Risk of stale rules	Real: “set it and forget it” leaves bad rules active for months	Low, but trades that risk for dependence on a well-monitored model
Cost to start	Low, you can start today with no new infrastructure	High, requires data volume, context instrumentation, and model monitoring

Manual rules win on transparency and simplicity. Contextual bandits win on scale and on catching patterns nobody would have thought to write as a rule. In practice, many mature operations run both: rules for the few obvious, high-stakes cases, and a contextual bandit for the wider space of combinations where human intuition cannot keep up.

When a Contextual Bandit Is NOT the Right Tool

The part most vendor marketing skips: there are two common, legitimate reasons to avoid a contextual bandit even when it looks technically superior.

The first reason is volume. A contextual bandit has to learn a policy for every relevant combination of context, not just one overall average. If your traffic is barely enough to close out a simple A/B test (see the sample size calculator for a sense of how much that usually takes), it almost certainly is not enough to train a model with several context variables at once. A data-starved model tends to “learn” patterns that are really just noise, and it will hand back worse decisions than a simple rule or a traditional A/B test would, not better ones.

The second reason is explainability. Small teams, and teams that answer to non-technical stakeholders, often need to justify decisions in one sentence: “we switched the CTA because B converted 30% more, at significance.” That sentence exists cleanly for an A/B test. It does not exist the same way for a contextual bandit, whose decision for a specific visitor is the product of a combination of learned weights, hard to summarize without oversimplifying. If your organization’s governance requires that kind of simple justification for every change, the cost of explaining a contextual bandit can outweigh the performance gain it delivers.

There is a third, more operational warning sign: a contextual bandit requires continuous monitoring of the model itself, not just of the outcome. A model that stops receiving quality data (say, a context source breaks silently) can keep making confident decisions, just wrong ones, without anyone noticing, because there is no longer a simple dashboard of “A converted X%, B converted Y%” to check against.

Tools and Platforms Offering Contextual Bandits Today

It is worth separating vendors that ship contextual bandits ready to use, as part of an experimentation or personalization platform, from those that offer only the machine-learning building block for you to assemble your own.

Tool	What it offers	Note
Optimizely	A Contextual Bandits feature inside its experimentation and personalization platform, described by the company itself as a way to deliver the most effective experience per visitor	Positioned as part of the personalization offering, not a standalone product
Dynamic Yield	Combines A/B/n testing, multi-armed bandit allocation, and AI-driven recommendations in the same omnichannel personalization platform	Strong focus on content and product recommendation
Datadog Experiments (formerly Eppo)	Contextual bandits integrated with experiment analysis, with a holdout strategy to measure the bandit’s real impact on business metrics	Eppo was acquired by Datadog; technical bandit documentation remains available in the product docs
Vowpal Wabbit	Open-source machine-learning library that implements contextual bandits and powers many custom in-house solutions	Requires engineering or ML capacity to integrate; not a ready-to-use marketing tool
Azure Personalizer	Microsoft’s contextual-bandit-based service (running on Vowpal Wabbit), now being retired, with shutdown confirmed for October 1, 2026	A useful reminder of the risk of building on a specific cloud vendor’s managed service; Microsoft itself recommends migrating to the open-source microsoft/learning-loop

The Azure Personalizer case is a useful reminder: a contextual bandit is not just an algorithm, it is also a long-term maintenance commitment, whether with a vendor or with an open-source stack your own team will have to operate.

Automate This with Donnu

A contextual bandit asks for two things most CRO operations do not yet have in place: enough data volume to train a policy per context, and bandit infrastructure running reliably, with monitoring of the model itself. Until that is realistic for your team, Donnu already solves the part that captures most of the upside of a plain bandit: traffic allocation that shifts on its own toward the variation that is actually performing, backed by honest Bayesian statistics, with no policy model for you to train or maintain.

Start a free 14-day trial and see dynamic allocation working on your own traffic. To see the full picture before choosing between the three approaches, read Multi-Armed Bandits vs. A/B Testing, and for the statistical foundation underneath all of them, see our Bayesian A/B testing guide and statistical significance guide.

Read this in Portuguese: O Que É um Bandit Contextual.

References

• Optimizely. Contextual bandits: the next step in personalization. optimizely.com/insights
• Eppo (now Datadog Experiments). Introducing: Eppo Contextual Bandits and Example use cases for Contextual Bandits. docs.geteppo.com/contextual-bandits
• Microsoft Learn. Reinforcement learning in Personalizer and the service’s retirement notice. learn.microsoft.com/azure/ai-services/personalizer
• Dynamic Yield. Beyond A/B testing: Multi-armed bandit experiments. dynamicyield.com
• VWO. Understanding the Working of Multi-Armed Bandit in VWO. help.vwo.com