# A Survey on Practical Applications of Multi-Armed and Contextual Bandits

arXiv: https://arxiv.org/abs/1904.10040 · HTML: https://mlsanity.com/paper/1904.10040/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2019.

## Built on by (most cited first, 30 of 45)

- [Efficient Prompt Optimization Through the Lens of Best Arm Identification](https://mlsanity.com/paper/2402.09723/) (2024)
- [Leveraging Good Representations in Linear Contextual Bandits](https://mlsanity.com/paper/2104.03781/) (2021)
- [Adaptive LLM Routing under Budget Constraints](https://mlsanity.com/paper/2508.21141/) (2025)
- [Batched Neural Bandits](https://mlsanity.com/paper/2102.13028/) (2021)
- [Differentially Private Stochastic Linear Bandits: (Almost) for Free](https://mlsanity.com/paper/2207.03445/) (2022)
- [Apollo: Transferable Architecture Exploration](https://mlsanity.com/paper/2102.01723/) (2021)
- [Solving Multi-Arm Bandit Using a Few Bits of Communication](https://mlsanity.com/paper/2111.06067/) (2021)
- [A Map of Bandits for E-commerce](https://mlsanity.com/paper/2107.00680/) (2021)
- [Contextual Restless Multi-Armed Bandits with Application to Demand Response Decision-Making](https://mlsanity.com/paper/2403.15640/) (2024)
- [Reinforcement Learning in Practice: Opportunities and Challenges](https://mlsanity.com/paper/2202.11296/) (2022)
- [Provably and Practically Efficient Neural Contextual Bandits](https://mlsanity.com/paper/2206.00099/) (2022)
- [Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Bandits](https://mlsanity.com/paper/2402.04933/) (2024)
- [Optimal Epidemic Control as a Contextual Combinatorial Bandit with Budget](https://mlsanity.com/paper/2106.15808/) (2021)
- [Integrating Reward Maximization and Population Estimation: Sequential Decision-Making for Internal Revenue Service Audit Selection](https://mlsanity.com/paper/2204.11910/) (2022)
- [Scalable Representation Learning in Linear Contextual Bandits with Constant Regret Guarantees](https://mlsanity.com/paper/2210.13083/) (2022)
- [Kolmogorov-Smirnov Test-Based Actively-Adaptive Thompson Sampling for Non-Stationary Bandits](https://mlsanity.com/paper/2105.14586/) (2021)
- [Dealing with Expert Bias in Collective Decision-Making](https://mlsanity.com/paper/2106.13539/) (2021)
- [Reward Model Routing in Alignment](https://mlsanity.com/paper/2510.02850/) (2025)
- [Remote Contextual Bandits](https://mlsanity.com/paper/2202.05182/) (2022)
- [A bandit-learning approach to multifidelity approximation](https://mlsanity.com/paper/2103.15342/) (2021)
- [Incentivized Bandit Learning with Self-Reinforcing User Preferences](https://mlsanity.com/paper/2105.08869/) (2021)
- [Bridging Adversarial and Nonstationary Multi-armed Bandit](https://mlsanity.com/paper/2201.01628/) (2022)
- [Contextual Decision Trees](https://mlsanity.com/paper/2207.06355/) (2022)
- [Conditionally Risk-Averse Contextual Bandits](https://mlsanity.com/paper/2210.13573/) (2022)
- [Leveraging heterogeneous spillover in maximizing contextual bandit rewards](https://mlsanity.com/paper/2310.10259/) (2023)
- [Variational Quantum Optimization with Continuous Bandits](https://mlsanity.com/paper/2502.04021/) (2025)
- [Graph Learning Is Suboptimal in Causal Bandits](https://mlsanity.com/paper/2510.16811/) (2025)
- [Risk averse non-stationary multi-armed bandits](https://mlsanity.com/paper/2109.13977/) (2021)
- [Fast and Interpretable Consensus Clustering via Minipatch Learning](https://mlsanity.com/paper/2110.02388/) (2021)
- [Multi-Armed Bandits in Brain-Computer Interfaces](https://mlsanity.com/paper/2205.09584/) (2022)

## Abstract (arXiv)

In recent years, multi-armed bandit (MAB) framework has attracted a lot of attention in various applications, from recommender systems and information retrieval to healthcare and finance, due to its stellar performance combined with certain attractive properties, such as learning from less feedback. The multi-armed bandit field is currently flourishing, as novel problem settings and algorithms motivated by various practical applications are being introduced, building on top of the classical bandit problem. This article aims to provide a comprehensive review of top recent developments in multiple real-life applications of the multi-armed bandit. Specifically, we introduce a taxonomy of common MAB-based applications and summarize state-of-art for each of those domains. Furthermore, we identify important current trends and provide new perspectives pertaining to the future of this exciting and fast-growing field.