# Stochastic Bandits with Context Distributions

arXiv: https://arxiv.org/abs/1906.02685 · HTML: https://mlsanity.com/paper/1906.02685/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2019.

## Built on by (most cited first, 18 of 18)

- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Misspecified Gaussian Process Bandit Optimization](https://mlsanity.com/paper/2111.05008/) (2021)
- [Neural Contextual Bandits without Regret](https://mlsanity.com/paper/2107.03144/) (2021)
- [Robust Bandit Learning with Imperfect Context](https://mlsanity.com/paper/2102.05018/) (2021)
- [Stochastic Conservative Contextual Linear Bandits](https://mlsanity.com/paper/2203.15629/) (2022)
- [BONSAI: Structure-exploiting robust Bayesian optimization for networked black-box systems under uncertainty](https://mlsanity.com/paper/2510.03893/) (2025)
- [Online learning in bandits with predicted context](https://mlsanity.com/paper/2307.13916/) (2023)
- [A Parametric Contextual Online Learning Theory of Brokerage](https://mlsanity.com/paper/2407.01566/) (2024)
- [Robust Bayesian Satisficing](https://mlsanity.com/paper/2308.08291/) (2023)
- [Stochastic Bayesian Optimization with Unknown Continuous Context Distribution via Kernel Density Estimation](https://mlsanity.com/paper/2312.10423/) (2023)
- [Wasserstein Distributionally Robust Bayesian Optimization with Continuous Context](https://mlsanity.com/paper/2503.20341/) (2025)
- [Distributed Stochastic Bandit Learning with Delayed Context Observation](https://mlsanity.com/paper/2207.14391/) (2022)
- [Distributed Multi-Task Learning for Stochastic Bandits with Context Distribution and Stage-wise Constraints](https://mlsanity.com/paper/2401.11563/) (2024)
- [Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysis](https://mlsanity.com/paper/2401.11565/) (2024)
- [Federated Learning for Heterogeneous Bandits with Unobserved Contexts](https://mlsanity.com/paper/2303.17043/) (2023)
- [Efficient Generalized Low-Rank Tensor Contextual Bandits](https://mlsanity.com/paper/2311.01771/) (2023)
- [Learn to Bid as a Price-Maker Wind Power Producer](https://mlsanity.com/paper/2503.16107/) (2025)
- [Towards Fully Automated Decision-Making Systems for Greenhouse Control: Challenges and Opportunities](https://mlsanity.com/paper/2503.21640/) (2025)

## Abstract (arXiv)

We introduce a stochastic contextual bandit model where at each time step the environment chooses a distribution over a context set and samples the context from this distribution. The learner observes only the context distribution while the exact context realization remains hidden. This allows for a broad range of applications where the context is stochastic or when the learner needs to predict the context. We leverage the UCB algorithm to this setting and show that it achieves an order-optimal high-probability bound on the cumulative regret for linear and kernelized reward functions. Our results strictly generalize previous work in the sense that both our model and the algorithm reduce to the standard setting when the environment chooses only Dirac delta distributions and therefore provides the exact context to the learner. We further analyze a variant where the learner observes the realized context after choosing the action. Finally, we demonstrate the proposed method on synthetic and real-world datasets.