# Contextual Bandits with Stochastic Experts

arXiv: https://arxiv.org/abs/1802.08737 · HTML: https://mlsanity.com/paper/1802.08737/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2018.

## Built on by (most cited first, 5 of 5)

- [Information Capacity Regret Bounds for Bandits with Mediator Feedback](https://mlsanity.com/paper/2402.10282/) (2024)
- [Information-Theoretic Regret Bounds for Bandits with Fixed Expert Advice](https://mlsanity.com/paper/2303.08102/) (2023)
- [Pure Exploration under Mediators' Feedback](https://mlsanity.com/paper/2308.15552/) (2023)
- [Bandits with Stochastic Experts: Constant Regret, Empirical Experts and Episodes](https://mlsanity.com/paper/2107.03263/) (2021)
- [Online Learning for Multi-Layer Hierarchical Inference under Partial and Policy-Dependent Feedback](https://mlsanity.com/paper/2603.04247/) (2026)

## Abstract (arXiv)

We consider the problem of contextual bandits with stochastic experts, which is a variation of the traditional stochastic contextual bandit with experts problem. In our problem setting, we assume access to a class of stochastic experts, where each expert is a conditional distribution over the arms given a context. We propose upper-confidence bound (UCB) algorithms for this problem, which employ two different importance sampling based estimators for the mean reward for each expert. Both these estimators leverage information leakage among the experts, thus using samples collected under all the experts to estimate the mean reward of any given expert. This leads to instance dependent regret bounds of Oleft(λ(pmbμ)Mlog T/Delta right), where λ(pmbμ) is a term that depends on the mean rewards of the experts, Delta is the smallest gap between the mean reward of the optimal expert and the rest, and M quantifies the information leakage among the experts. We show that under some assumptions λ(pmbμ) is typically O(log N), where N is the number of experts. We implement our algorithm with stochastic experts generated from cost-sensitive classification oracles and show superior empirical performance on real-world datasets, when compared to other state of the art contextual bandit algorithms.