# Efficient Exploration in Binary and Preferential Bayesian Optimization

arXiv: https://arxiv.org/abs/2110.09361 · HTML: https://mlsanity.com/paper/2110.09361/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [Batched Large-scale Bayesian Optimization in High-dimensional Spaces](https://mlsanity.com/paper/1706.01445/) (2017)
- [Constrained Bayesian Optimization with Noisy Experiments](https://mlsanity.com/paper/1706.07094/) (2017)
- [Hilbert Space Methods for Reduced-Rank Gaussian Process Regression](https://mlsanity.com/paper/1401.5508/) (2014)
- [Preferential Bayesian Optimization](https://mlsanity.com/paper/1704.03651/) (2017)
- [Pathwise Conditioning of Gaussian Processes](https://mlsanity.com/paper/2011.04026/) (2020)
- [Multi-dueling Bandits with Dependent Arms](https://mlsanity.com/paper/1705.00253/) (2017)
- [Practical Hilbert space approximate Bayesian Gaussian processes for probabilistic programming](https://mlsanity.com/paper/2004.11408/) (2020)
- [Top-k Ranking Bayesian Optimization](https://mlsanity.com/paper/2012.10688/) (2020)
- [Preferential Batch Bayesian Optimization](https://mlsanity.com/paper/2003.11435/) (2020)
- [Preferential Bayesian optimisation with Skew Gaussian Processes](https://mlsanity.com/paper/2008.06677/) (2020)
- [Bayesian Optimization with Binary Auxiliary Information](https://mlsanity.com/paper/1906.07277/) (2019)
- [When is it Better to Compare than to Score?](https://mlsanity.com/paper/1406.6618/) (2014)
- [Heteroscedastic Bayesian Optimisation for Stochastic Model Predictive Control](https://mlsanity.com/paper/2010.00202/) (2020)
- [Sequential Preference-Based Optimization](https://mlsanity.com/paper/1801.02788/) (2018)
- [A Stratified Analysis of Bayesian Optimization Methods](https://mlsanity.com/paper/1603.09441/) (2016)

## Built on by (most cited first, 5 of 5)

- [Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes](https://mlsanity.com/paper/2302.01513/) (2023)
- [Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses](https://mlsanity.com/paper/2306.13104/) (2023)
- [A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation](https://mlsanity.com/paper/2302.03570/) (2023)
- [Contextual Bayesian optimization with binary outputs](https://mlsanity.com/paper/2111.03447/) (2021)
- [DT-PBO: an Interpretable Tree-based Surrogate Model for Preferential Bayesian Optimization](https://mlsanity.com/paper/2512.14263/) (2025)

## Abstract (arXiv)

Bayesian optimization (BO) is an effective approach to optimize expensive black-box functions, that seeks to trade-off between exploitation (selecting parameters where the maximum is likely) and exploration (selecting parameters where we are uncertain about the objective function). In many real-world situations, direct measurements of the objective function are not possible, and only binary measurements such as success/failure or pairwise comparisons are available. To perform efficient exploration in this setting, we show that it is important for BO algorithms to distinguish between different types of uncertainty: epistemic uncertainty, about the unknown objective function, and aleatoric uncertainty, which comes from noisy observations and cannot be reduced. In effect, only the former is important for efficient exploration. Based on this, we propose several new acquisition functions that outperform state-of-the-art heuristics in binary and preferential BO, while being fast to compute and easy to implement. We then generalize these acquisition rules to batch learning, where multiple queries are performed simultaneously.