mlsanityData to 7 Oct 2026

A Survey on Practical Applications of Multi-Armed and Contextual Bandits

arXiv 1904.10040 ↗First posted 2 Apr 2019cs.LG stat.MLcited by 44

Not on the map. Why?

Abstract (arXiv)

In recent years, multi-armed bandit (MAB) framework has attracted a lot of attention in various applications, from recommender systems and information retrieval to healthcare and finance, due to its stellar performance combined with certain attractive properties, such as learning from less feedback. The multi-armed bandit field is currently flourishing, as novel problem settings and algorithms motivated by various practical applications are being introduced, building on top of the classical bandit problem. This article aims to provide a comprehensive review of top recent developments in multiple real-life applications of the multi-armed bandit. Specifically, we introduce a taxonomy of common MAB-based applications and summarize state-of-art for each of those domains. Furthermore, we identify important current trends and provide new perspectives pertaining to the future of this exciting and fast-growing field.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 11; 2023: 4; 2024: 4; 2025: 8; 2026: 4.

44 papers.

Sort:

By area:

  1. 2024Efficient Prompt Optimization Through the Lens of Best Arm Identificationcited by 35
  2. 2021Leveraging Good Representations in Linear Contextual Banditscited by 29
  3. 2021Batched Neural Banditscited by 22
  4. 2025Adaptive LLM Routing under Budget Constraintscited by 21
  5. 2022Differentially Private Stochastic Linear Bandits: (Almost) for Freecited by 19
  6. 2021Apollo: Transferable Architecture Explorationcited by 17
  7. 2021Solving Multi-Arm Bandit Using a Few Bits of Communicationcited by 16
  8. 2021A Map of Bandits for E-commercecited by 9
  9. 2022Provably and Practically Efficient Neural Contextual Banditscited by 6
  10. 2024Contextual Restless Multi-Armed Bandits with Application to Demand Response Decision-Makingcited by 6
  11. 2021Optimal Epidemic Control as a Contextual Combinatorial Bandit with Budgetcited by 5
  12. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 5
32 more
  1. 2022Integrating Reward Maximization and Population Estimation: Sequential Decision-Making for Internal Revenue Service Audit Selectioncited by 5
  2. 2022Scalable Representation Learning in Linear Contextual Bandits with Constant Regret Guaranteescited by 5
  3. 2024Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Banditscited by 5
  4. 2021Kolmogorov-Smirnov Test-Based Actively-Adaptive Thompson Sampling for Non-Stationary Banditscited by 4
  5. 2021Dealing with Expert Bias in Collective Decision-Makingcited by 4
  6. 2022Remote Contextual Banditscited by 3
  7. 2025Reward Model Routing in Alignmentcited by 3
  8. 2021A bandit-learning approach to multifidelity approximationcited by 2
  9. 2021Incentivized Bandit Learning with Self-Reinforcing User Preferencescited by 2
  10. 2022Bridging Adversarial and Nonstationary Multi-armed Banditcited by 2
  11. 2022Contextual Decision Treescited by 2
  12. 2022Conditionally Risk-Averse Contextual Banditscited by 2
  13. 2023Leveraging heterogeneous spillover in maximizing contextual bandit rewardscited by 2
  14. 2025Variational Quantum Optimization with Continuous Banditscited by 2
  15. 2025Graph Learning Is Suboptimal in Causal Banditscited by 2
  16. 2021Risk averse non-stationary multi-armed banditscited by 1
  17. 2021Fast and Interpretable Consensus Clustering via Minipatch Learningcited by 1
  18. 2022Multi-Armed Bandits in Brain-Computer Interfacescited by 1
  19. 2023Causal Inference out of Control: Estimating the Steerability of Consumptioncited by 1
  20. 2023Non-Stationary Contextual Bandit Learning via Neural Predictive Ensemble Samplingcited by 1
  21. 2023New Classes of the Greedy-Applicable Arm Feature Distributions in the Sparse Linear Bandit Problemcited by 1
  22. 2025LNUCB-TA: Linear-nonlinear Hybrid Bandit Learning with Temporal Attentioncited by 1
  23. 2025Asymptotically Optimal Linear Best Feasible Arm Identification with Fixed Budgetcited by 1
  24. 2021Etat de l'art sur l'application des bandits multi-brascited by 0
  25. 2022Machine Learning Prescriptive Canvas for Optimizing Business Outcomescited by 0
  26. 2024Survival Multiarmed Bandits with Bootstrapping Methodscited by 0
  27. 2025Hierarchical Adversarial Bandits for Online Configuration Optimizationcited by 0
  28. 2025Consistency of Selection Strategies for Fraud Detectioncited by 0
  29. 2026A Simple Reduction Scheme for Constrained Contextual Bandits with Adversarial Contexts via Regressioncited by 0
  30. 2026Beyond Static Bias: Adaptive Multi-Fidelity Bandits with Improving Proxiescited by 0
  31. 2026Safety by Design: Realized-Cost Constraints for Contextual Bandits with Continuous Actionscited by 0
  32. 2026Expected Sample Complexity in Multi-Armed Banditscited by 0