Multi-armed bandit testing is a method for running experiments that shifts traffic toward the better-performing option while the test is still live, instead of splitting traffic evenly until a fixed end date. The name comes from the "one-armed bandit" slot machine: if you have several slot machines with unknown payout rates, how do you decide which to keep pulling to maximize total winnings?
What changed in 2026
- Contextual bandits — which use user or situational features to personalize the "best arm" per user rather than picking one global winner — became standard in recommendation engines and content-ranking use cases, well beyond their original marketing-test roots.
- Thompson sampling solidified as the default algorithm in most modern experimentation platforms, largely displacing simpler epsilon-greedy approaches for production use.
- Bandit and fixed-horizon testing converged in tooling — most major platforms now let teams choose per-experiment rather than committing to one paradigm platform-wide.
The exploration vs exploitation tradeoff
Every bandit algorithm is answering the same question: given what you know so far, how much traffic do you send to the option that currently looks best (exploitation) versus how much do you send to options you are still uncertain about (exploration)? Too much exploitation risks missing a better option that had a rough early sample. Too much exploration wastes traffic on options you already have enough evidence are worse.
The three common algorithms
Epsilon-greedy. Send most traffic to the current best-performing arm, and a small fixed percentage (epsilon) to random exploration. Simple to implement and reason about, but not very efficient — it explores the same amount regardless of how much uncertainty actually remains.
Upper Confidence Bound (UCB). Favor arms with high estimated performance or high uncertainty — arms you have not tried much get a confidence "bonus" that fades as more data comes in. More efficient than epsilon-greedy, more complex to implement.
Thompson sampling. Maintain a probability distribution over each arm's true performance, and sample from those distributions to decide where to send the next unit of traffic. It naturally balances exploration and exploitation without a manually tuned parameter, which is a large part of why it became the default.
Bandit testing vs standard A/B testing
| Factor |
Bandit testing |
Fixed A/B testing |
| Traffic allocation |
Shifts dynamically during the test |
Fixed split for the full duration |
| Optimizes for |
Minimizing regret (losses during the test) |
A clean, statistically confident final answer |
| Best for |
High-traffic, short-lived, low-stakes decisions |
High-stakes, infrequent, or hard-to-reverse decisions |
| Statistical clarity |
Weaker, harder to explain post-hoc |
Strong, well-understood significance testing |
| Setup complexity |
Higher; needs careful reward definition |
Lower; standard tooling widely available |
When to actually use a bandit
Bandits earn their complexity when the cost of showing a worse option during the test is real and traffic is abundant enough for the algorithm to learn quickly — a homepage promo banner, a subject line test with a fast decay window, a real-time content ranking decision. They are a poor fit when you need a clean, defensible causal answer for a decision you plan to live with for a long time, such as a major pricing or product change, where the interpretability of a fixed A/B test matters more than minimizing in-test regret.
Common pitfalls
- Using a bandit for a decision that needs stakeholder-legible statistics — bandit output is harder to explain than a p-value and confidence interval.
- Defining the reward signal poorly. A bandit optimizing for clicks will happily learn to favor clickbait if that is the only signal it is given.
- Assuming a bandit needs less traffic than an A/B test. It still needs meaningful volume to learn reliably; it just spends that volume differently.
FAQ
Is a bandit test faster than an A/B test?
It reduces regret during the test (fewer users see the worse option), but it does not necessarily reach a confident conclusion faster — that still depends on the effect size and traffic volume.
Can I switch a running A/B test into a bandit?
Not cleanly. The statistical assumptions differ enough that switching mid-test invalidates guarantees on both sides. Decide the approach before launch.
Do contextual bandits replace recommendation engines?
No — they are often a component inside one, particularly for the cold-start and exploration problem, not a replacement for the ranking model itself.
Where to go next