# Neural Contextual Bandits with UCB-based Exploration

arXiv: https://arxiv.org/abs/1911.04462 · HTML: https://mlsanity.com/paper/1911.04462/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 6 of 6)

- [Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions](https://mlsanity.com/paper/2203.02605/) (2022)
- [Context Uncertainty in Contextual Bandits with Applications to Recommender Systems](https://mlsanity.com/paper/2202.00805/) (2022)
- [X2T: Training an X-to-Text Typing Interface with Online Learning from User Feedback](https://mlsanity.com/paper/2203.02072/) (2022)
- [Language Model Prompt Selection via Simulation Optimization](https://mlsanity.com/paper/2404.08164/) (2024)
- [Towards Domain Adaptive Neural Contextual Bandits](https://mlsanity.com/paper/2406.09564/) (2024)
- [Automated Customization of On-Thing Inference for Quality-of-Experience Enhancement](https://mlsanity.com/paper/2112.06918/) (2021)

## Abstract (arXiv)

We study the stochastic contextual bandit problem, where the reward is generated from an unknown function with additive noise. No assumption is made about the reward function other than boundedness. We propose a new algorithm, NeuralUCB, which leverages the representation power of deep neural networks and uses a neural network-based random feature mapping to construct an upper confidence bound (UCB) of reward for efficient exploration. We prove that, under standard assumptions, NeuralUCB achieves tilde O(√T) regret, where T is the number of rounds. To the best of our knowledge, it is the first neural network-based contextual bandit algorithm with a near-optimal regret guarantee. We also show the algorithm is empirically competitive against representative baselines in a number of benchmarks.