# UCB for Large-Scale Pure Exploration: Beyond Sub-Gaussianity

arXiv: https://arxiv.org/abs/2511.22273 · HTML: https://mlsanity.com/paper/2511.22273/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 56 references that are in the map.
- First posted on arXiv 27 Nov 2025.

## Builds on (most cited first, 16 of 16)

- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Optimal Best Arm Identification with Fixed Confidence](https://mlsanity.com/paper/1602.04589/) (2016)
- [lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Bandits](https://mlsanity.com/paper/1312.7308/) (2013)
- [Simple Bayesian Algorithms for Best Arm Identification](https://mlsanity.com/paper/1602.08448/) (2016)
- [The Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms](https://mlsanity.com/paper/2002.10121/) (2020)
- [Review on Ranking and Selection: A New Perspective](https://mlsanity.com/paper/2008.00249/) (2020)
- [Bridging the gap between regret minimization and best arm identification, with application to A/B tests](https://mlsanity.com/paper/1810.04088/) (2018)
- [Optimal Best-Arm Identification Methods for Tail-Risk Measures](https://mlsanity.com/paper/2008.07606/) (2020)
- [The Fragility of Optimized Bandit Algorithms](https://mlsanity.com/paper/2109.13595/) (2021)
- [Catoni-style confidence sequences for heavy-tailed mean estimation](https://mlsanity.com/paper/2202.01250/) (2022)
- [Achieving the Pareto Frontier of Regret Minimization and Best Arm Identification in Multi-Armed Bandits](https://mlsanity.com/paper/2110.08627/) (2021)
- [Fast and Regret Optimal Best Arm Identification: Fundamental Limits and Low-Complexity Algorithms](https://mlsanity.com/paper/2309.00591/) (2023)
- [On Finding the Largest Mean Among Many](https://mlsanity.com/paper/1306.3917/) (2013)
- [Dual-Directed Algorithm Design for Efficient Pure Exploration](https://mlsanity.com/paper/2310.19319/) (2023)
- [Efficient Ranking and Selection in Parallel Computing Environments](https://mlsanity.com/paper/1506.04986/) (2015)
- [Asymptotic Validity of the Bayes-Inspired Indifference Zone Procedure: The Non-Normal Known Variance Case](https://mlsanity.com/paper/1508.07720/) (2015)

## Abstract (arXiv)

Selecting the best alternative from a finite set is the central objective of ranking and selection (R&S) and best arm identification (BAI). Traditional R&S or BAI approaches have predominantly relied on Gaussian or sub-Gaussian assumptions on the performance distributions of all alternatives, which limit their applicability to non-sub-Gaussian---especially heavy-tailed---problems. The need to move beyond sub-Gaussianity may become even more critical in large-scale problems, which tend to be especially sensitive to distributional specifications. In this paper, motivated by the widespread use of upper confidence bound (UCB) algorithms in sequential decision making, we investigate their performance in large-scale, non-sub-Gaussian R&S settings. We consider the simplest category of UCB algorithms, where the UCB value for each alternative is defined as the sample mean plus an exploration bonus that depends only on its own sample size. We abstract this into a meta-UCB algorithm and propose letting it select the alternative with the largest sample size as the best upon stopping. For this meta-UCB algorithm, we first derive a distribution-free lower bound on the probability of correct selection. Building on this bound, we first study the indifference-zone formulation and show that the meta-UCB algorithm---and therefore a broad class of UCB algorithms---achieves sample optimality as long as the variances are uniformly bounded. We then extend sample optimality to a non-indifference-zone configuration with polynomially shrinking mean gaps under the same variance assumption. These results demonstrate the applicability of UCB algorithms to large-scale R&S problems with non-sub-Gaussian distributions. Numerical experiments support our results and provide additional insights into the comparative behaviors of UCB algorithms within and beyond our meta-UCB framework.