# Adapting multi-armed bandits policies to contextual bandits scenarios

arXiv: https://arxiv.org/abs/1811.04383 · HTML: https://mlsanity.com/paper/1811.04383/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2018.

## Built on by (most cited first, 8 of 8)

- [AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning](https://mlsanity.com/paper/2107.08147/) (2021)
- [Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation](https://mlsanity.com/paper/2403.13578/) (2024)
- [Energy Regularized RNNs for Solving Non-Stationary Bandit Problems](https://mlsanity.com/paper/2303.06552/) (2023)
- [Harnessing the Power of Federated Learning in Federated Contextual Bandits](https://mlsanity.com/paper/2312.16341/) (2023)
- [JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization](https://mlsanity.com/paper/2606.11425/) (2026)
- [A Hybrid Strategy for Chat Transcript Summarization](https://mlsanity.com/paper/2402.01510/) (2024)
- [Optimizing Warfarin Dosing Using Contextual Bandit: An Offline Policy Learning and Evaluation Method](https://mlsanity.com/paper/2402.11123/) (2024)
- [MBExplainer: Multilevel bandit-based explanations for downstream models with augmented graph embeddings](https://mlsanity.com/paper/2411.00287/) (2024)

## Abstract (arXiv)

This work explores adaptations of successful multi-armed bandits policies to the online contextual bandits scenario with binary rewards using binary classification algorithms such as logistic regression as black-box oracles. Some of these adaptations are achieved through bootstrapping or approximate bootstrapping, while others rely on other forms of randomness, resulting in more scalable approaches than previous works, and the ability to work with any type of classification algorithm. In particular, the Adaptive-Greedy algorithm shows a lot of promise, in many cases achieving better performance than upper confidence bound and Thompson sampling strategies, at the expense of more hyperparameters to tune.