# A Survey on Contextual Multi-armed Bandits

arXiv: https://arxiv.org/abs/1508.03326 · HTML: https://mlsanity.com/paper/1508.03326/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2015.

## Built on by (most cited first, 30 of 34)

- [Lero: A Learning-to-Rank Query Optimizer](https://mlsanity.com/paper/2302.06873/) (2023)
- [Metadata-based Multi-Task Bandits with Bayesian Hierarchical Models](https://mlsanity.com/paper/2108.06422/) (2021)
- [Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook](https://mlsanity.com/paper/2210.13623/) (2022)
- [Personalized Reward Learning with Interaction-Grounded Learning (IGL)](https://mlsanity.com/paper/2211.15823/) (2022)
- [Learning Action Embeddings for Off-Policy Evaluation](https://mlsanity.com/paper/2305.03954/) (2023)
- [Contextual Restless Multi-Armed Bandits with Application to Demand Response Decision-Making](https://mlsanity.com/paper/2403.15640/) (2024)
- [LASeR: Learning to Adaptively Select Reward Models with Multi-Armed Bandits](https://mlsanity.com/paper/2410.01735/) (2024)
- [Knowledge Infused Policy Gradients with Upper Confidence Bound for Relational Bandits](https://mlsanity.com/paper/2106.13895/) (2021)
- [Pricing with Contextual Elasticity and Heteroscedastic Valuation](https://mlsanity.com/paper/2312.15999/) (2023)
- [Dealing with Expert Bias in Collective Decision-Making](https://mlsanity.com/paper/2106.13539/) (2021)
- [Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learning](https://mlsanity.com/paper/2110.15501/) (2021)
- [Coordinated Attacks against Contextual Bandits: Fundamental Limits and Defense Mechanisms](https://mlsanity.com/paper/2201.12700/) (2022)
- [Reward Model Routing in Alignment](https://mlsanity.com/paper/2510.02850/) (2025)
- [Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear Bandits](https://mlsanity.com/paper/2306.14872/) (2023)
- [Approximate information for efficient exploration-exploitation strategies](https://mlsanity.com/paper/2307.01563/) (2023)
- [Multi-Excitation Projective Simulation with a Many-Body Physics Inspired Inductive Bias](https://mlsanity.com/paper/2402.10192/) (2024)
- [A Scalable Recommendation Engine for New Users and Items](https://mlsanity.com/paper/2209.06128/) (2022)
- [Sequential Decision Making on Unmatched Data using Bayesian Kernel Embeddings](https://mlsanity.com/paper/2210.13692/) (2022)
- [Hierarchical Upper Confidence Bounds for Constrained Online Learning](https://mlsanity.com/paper/2410.17216/) (2024)
- [Learning to Orchestrate Agents under Uncertainty](https://mlsanity.com/paper/2605.27073/) (2026)
- [JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization](https://mlsanity.com/paper/2606.11425/) (2026)
- [Design Principles of Robust Multi-Armed Bandit Framework in Video Recommendations](https://mlsanity.com/paper/2310.01419/) (2023)
- [Identifiable Latent Bandits: Leveraging observational data for personalized decision-making](https://mlsanity.com/paper/2407.16239/) (2024)
- [Reciprocal Learning](https://mlsanity.com/paper/2408.06257/) (2024)
- [Enhancing Federated Graph Learning via Adaptive Fusion of Structural and Node Characteristics](https://mlsanity.com/paper/2412.18845/) (2024)
- [Information maximization for a broad variety of multi-armed bandit games](https://mlsanity.com/paper/2503.15962/) (2025)
- [In-Domain African Languages Translation Using LLMs and Multi-armed Bandits](https://mlsanity.com/paper/2505.15069/) (2025)
- [Scalable and Interpretable Contextual Bandits: A Literature Review and Retail Offer Prototype](https://mlsanity.com/paper/2505.16918/) (2025)
- [BanditWare: A Contextual Bandit-based Framework for Hardware Prediction](https://mlsanity.com/paper/2506.13730/) (2025)
- [Latent Preference Bandits](https://mlsanity.com/paper/2508.05367/) (2025)

## Abstract (arXiv)

In this survey we cover a few stochastic and adversarial contextual bandit algorithms. We analyze each algorithm's assumption and regret bound.