# Blind Exploration and Exploitation of Stochastic Experts

arXiv: https://arxiv.org/abs/2104.01078 · HTML: https://mlsanity.com/paper/2104.01078/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 24 references that are in the map.
- First posted on arXiv 2 Apr 2021.

## Builds on (most cited first, 5 of 5)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-armed Bandit Problem with Multiple Plays](https://mlsanity.com/paper/1506.00779/) (2015)
- [Ranking and combining multiple predictors without labeled data](https://mlsanity.com/paper/1303.3257/) (2013)
- [Unsupervised Ensemble Learning with Dependent Classifiers](https://mlsanity.com/paper/1510.05830/) (2015)
- [A Note on KL-UCB+ Policy for the Stochastic Bandit](https://mlsanity.com/paper/1903.07839/) (2019)

## Abstract (arXiv)

We present blind exploration and exploitation (BEE) algorithms for identifying the most reliable stochastic expert based on formulations that employ posterior sampling, upper-confidence bounds, empirical Kullback-Leibler divergence, and minmax methods for the stochastic multi-armed bandit problem. Joint sampling and consultation of experts whose opinions depend on the hidden and random state of the world becomes challenging in the unsupervised, or blind, framework as feedback from the true state is not available. We propose an empirically realizable measure of expert competence that can be inferred instantaneously using only the opinions of other experts. This measure preserves the ordering of true competences and thus enables joint sampling and consultation of stochastic experts based on their opinions on dynamically changing tasks. Statistics derived from the proposed measure is instantaneously available allowing both blind exploration-exploitation and unsupervised opinion aggregation. We discuss how the lack of supervision affects the asymptotic regret of BEE architectures that rely on UCB1, KL-UCB, MOSS, IMED, and Thompson sampling. We demonstrate the performance of different BEE algorithms empirically and compare them to their standard, or supervised, counterparts.