# Mastering the exploration-exploitation trade-off in Bayesian Optimization

arXiv: https://arxiv.org/abs/2305.08624 · HTML: https://mlsanity.com/paper/2305.08624/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 15 May 2023.

## Builds on (most cited first, 21 of 21)

- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [Bayesian Optimisation over Multiple Continuous and Categorical Inputs](https://mlsanity.com/paper/1906.08878/) (2019)
- [Joint Entropy Search for Maximally-Informed Bayesian Optimization](https://mlsanity.com/paper/2206.04771/) (2022)
- [Greed is Good: Exploration and Exploitation Trade-offs in Bayesian Optimisation](https://mlsanity.com/paper/1911.12809/) (2019)
- [TREGO: a Trust-Region Framework for Efficient Global Optimization](https://mlsanity.com/paper/2101.06808/) (2021)
- [Bayesian Optimization for Categorical and Category-Specific Continuous Inputs](https://mlsanity.com/paper/1911.12473/) (2019)
- [Randomised Gaussian Process Upper Confidence Bound for Bayesian Optimisation](https://mlsanity.com/paper/2006.04296/) (2020)
- [mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions](https://mlsanity.com/paper/1703.03373/) (2017)
- [Global optimization via inverse distance weighting and radial basis functions](https://mlsanity.com/paper/1906.06498/) (2019)
- [Fair and Green Hyperparameter Optimization via Multi-objective and Multiple Information Source Bayesian Optimization](https://mlsanity.com/paper/2205.08835/) (2022)
- [Generalizing Bayesian Optimization with Decision-theoretic Entropies](https://mlsanity.com/paper/2210.01383/) (2022)
- [Bayesian Optimization for Machine Learning : A Practical Guidebook](https://mlsanity.com/paper/1612.04858/) (2016)
- [Green Machine Learning via Augmented Gaussian Processes and Multi-Information Source Optimization](https://mlsanity.com/paper/2006.14233/) (2020)
- [A comparison of mixed-variables Bayesian optimization approaches](https://mlsanity.com/paper/2111.01533/) (2021)
- [PI is back! Switching Acquisition Functions in Bayesian Optimization](https://mlsanity.com/paper/2211.01455/) (2022)
- [Learning Optimal Control of Water Distribution Networks through Sequential Model-based Optimization](https://mlsanity.com/paper/2003.04268/) (2020)
- [Uncertainty quantification and exploration-exploitation trade-off in humans](https://mlsanity.com/paper/2102.07647/) (2021)

## Built on by (most cited first, 3 of 3)

- [Systematic Evaluation of Initial States and Exploration-Exploitation Strategies in PID Auto-Tuning: A Framework-Driven Approach Applied on Mobile Robots](https://mlsanity.com/paper/2505.03159/) (2025)
- [Hyperparameter Optimization of Constraint Programming Solvers](https://mlsanity.com/paper/2601.11389/) (2026)
- [Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding](https://mlsanity.com/paper/2602.00781/) (2026)

## Abstract (arXiv)

Gaussian Process based Bayesian Optimization is a well-known sample efficient sequential strategy for globally optimizing black-box, expensive, and multi-extremal functions. The role of the Gaussian Process is to provide a probabilistic approximation of the unknown function, depending on the sequentially collected observations, while an acquisition function drives the choice of the next solution to evaluate, balancing between exploration and exploitation, depending on the current Gaussian Process model. Despite the huge effort of the scientific community in defining effective exploration-exploitation mechanisms, we are still far away from the master acquisition function. This paper merges the most relevant results and insights from both algorithmic and human search strategies to propose a novel acquisition function, mastering the trade-off between explorative and exploitative choices, adaptively. We compare the proposed acquisition function on a number of test functions and against different state-of-the-art ones, which are instead based on prefixed or random scheduling between exploration and exploitation. A Pareto analysis is performed with respect to two (antagonistic) goals: convergence to the optimum and exploration capability. Results empirically prove that the proposed acquisition function is almost always Pareto optimal and also the most balanced trade-off between the two goals.