mlsanityData to 7 Oct 2026

Output-Weighted Sampling for Multi-Armed Bandits with Extreme Payoffs

arXiv 2102.10085 ↗First posted 19 Feb 2021cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

We present a new type of acquisition functions for online decision making in multi-armed and contextual bandit problems with extreme payoffs. Specifically, we model the payoff function as a Gaussian process and formulate a novel type of upper confidence bound (UCB) acquisition function that guides exploration towards the bandits that are deemed most relevant according to the variability of the observed rewards. This is achieved by computing a tractable likelihood ratio that quantifies the importance of the output relative to the inputs and essentially acts as an textitattention mechanism that promotes exploration of extreme rewards. We demonstrate the benefits of the proposed methodology across several synthetic benchmarks, as well as a realistic example involving noisy sensor network data. Finally, we provide a JAX library for efficient bandit optimization using Gaussian processes.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 17 of its 55 references that are in the map. First posted on arXiv 19 Feb 2021.

Builds on

17 of 55 references.

References · 17

  1. 2015Prioritized Experience Replaycited by 1,070
  2. 2015Automatic differentiation in machine learning: a surveycited by 846
  3. 2016Deep Exploration via Bootstrapped DQNcited by 518
  4. 2017A Tutorial on Thompson Samplingcited by 420
  5. 2014An Information-Theoretic Analysis of Thompson Samplingcited by 178
  6. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  7. 2020Efficiently Sampling Functions from Gaussian Process Posteriorscited by 125
  8. 2018Efficient Exploration through Bayesian Deep Q-Networkscited by 67
  9. 2017Improving the Expected Improvement Algorithmcited by 54
  10. 2018A sequential sampling strategy for extreme event statistics in nonlinear dynamical systemscited by 14
  11. 2016Thompson Sampling is Asymptotically Optimal in General Environmentscited by 8
  12. 2019Multi-fidelity classification using Gaussian processes: accelerating the prediction of large-scale computational modelscited by 8
5 more
  1. 2019Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samplescited by 7
  2. 2020Informative Path Planning for Extreme Anomaly Detection in Environment Exploration and Monitoringcited by 5
  3. 2019Adversarial Learning of a Sampler Based on an Unnormalized Distributioncited by 2
  4. 2020Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantificationcited by 2
  5. 2020Bayesian Optimization with Output-Weighted Optimal Samplingcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 1.

5 papers.

  1. 2022Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priorscited by 39
  2. 2022Discovering and forecasting extreme events via active learning in neural operatorscited by 32
  3. 2023Evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systemscited by 2
  4. 2024Active search for Bifurcationscited by 2
  5. 2023A generalized likelihood-weighted optimal sampling algorithm for rare-event probability quantificationcited by 0