# Deterministic Global Optimization of the Acquisition Function in Bayesian Optimization: To Do or Not To Do?

arXiv: https://arxiv.org/abs/2503.03625 · HTML: https://mlsanity.com/paper/2503.03625/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 75 references that are in the map.
- First posted on arXiv 5 Mar 2025.

## Builds on (most cited first, 30 of 30)

- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [High Dimensional Bayesian Optimisation and Bandits via Additive Models](https://mlsanity.com/paper/1503.01673/) (2015)
- [Batched Large-scale Bayesian Optimization in High-dimensional Spaces](https://mlsanity.com/paper/1706.01445/) (2017)
- [An Alternative View: When Does SGD Escape Local Minima?](https://mlsanity.com/paper/1802.06175/) (2018)
- [Maximizing acquisition functions for Bayesian optimization](https://mlsanity.com/paper/1805.10196/) (2018)
- [A Literature Survey of Benchmark Functions For Global Optimization Problems](https://mlsanity.com/paper/1308.4008/) (2013)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [BOCK : Bayesian Optimization with Cylindrical Kernels](https://mlsanity.com/paper/1806.01619/) (2018)
- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?](https://mlsanity.com/paper/2103.09655/) (2021)
- [Greed is Good: Exploration and Exploitation Trade-offs in Bayesian Optimisation](https://mlsanity.com/paper/1911.12809/) (2019)
- [SGD Converges to Global Minimum in Deep Learning via Star-convex Path](https://mlsanity.com/paper/1901.00451/) (2019)
- [Global Deterministic Optimization with Artificial Neural Networks Embedded](https://mlsanity.com/paper/1801.07114/) (2018)
- [Automatic Termination for Hyperparameter Optimization](https://mlsanity.com/paper/2104.08166/) (2021)
- [The Behavior and Convergence of Local Bayesian Optimization](https://mlsanity.com/paper/2305.15572/) (2023)
- [Bayesian Optimization with Exponential Convergence](https://mlsanity.com/paper/1604.01348/) (2016)
- [Constrained Discrete Black-Box Optimization using Mixed-Integer Programming](https://mlsanity.com/paper/2110.09569/) (2021)
- [Global Optimization of Gaussian processes](https://mlsanity.com/paper/2005.10902/) (2020)
- [The Vizier Gaussian Process Bandit Algorithm](https://mlsanity.com/paper/2408.11527/) (2024)
- [Multi-Objective Constrained Optimization for Energy Applications via Tree Ensembles](https://mlsanity.com/paper/2111.03140/) (2021)
- [Stopping Bayesian Optimization with Probabilistic Regret Bounds](https://mlsanity.com/paper/2402.16811/) (2024)
- [Global Optimization of Gaussian Process Acquisition Functions Using a Piecewise-Linear Kernel Approximation](https://mlsanity.com/paper/2410.16893/) (2024)
- [On Local Optimizers of Acquisition Functions in Bayesian Optimization](https://mlsanity.com/paper/1901.08350/) (2019)
- [Rapid Bayesian optimisation for synthesis of short polymer fiber materials](https://mlsanity.com/paper/1802.05841/) (2018)
- [Self-Adjusting Weighted Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2306.04262/) (2023)
- [Escaping local minima with derivative-free methods: a numerical investigation](https://mlsanity.com/paper/1812.11343/) (2018)
- [Computational and Exploratory Landscape Analysis of the GKLS Generator](https://mlsanity.com/paper/2304.08913/) (2023)
- [Optimal Initialization of Batch Bayesian Optimization](https://mlsanity.com/paper/2404.17997/) (2024)
- [Adaptive Bayesian Optimization Algorithm for Unpredictable Business Environments](https://mlsanity.com/paper/2401.11264/) (2024)

## Built on by (most cited first, 3 of 3)

- [A Guide to Bayesian Optimization in Bioprocess Engineering](https://mlsanity.com/paper/2508.10642/) (2025)
- [On Some Tunable Multi-fidelity Bayesian Optimization Frameworks](https://mlsanity.com/paper/2508.01013/) (2025)
- [An Efficient Spatial Branch-and-Bound Algorithm for Global Optimization of Gaussian Process Posterior Mean Functions](https://mlsanity.com/paper/2605.00855/) (2026)

## Abstract (arXiv)

Bayesian Optimization (BO) with Gaussian Processes relies on optimizing an acquisition function to determine sampling. We investigate the advantages and disadvantages of using a deterministic global solver (MAiNGO) compared to conventional local and stochastic global solvers (L-BFGS-B and multi-start, respectively) for the optimization of the acquisition function. For CPU efficiency, we set a time limit for MAiNGO, taking the best point as optimal. We perform repeated numerical experiments, initially using the Muller-Brown potential as a benchmark function, utilizing the lower confidence bound acquisition function; we further validate our findings with three alternative benchmark functions. Statistical analysis reveals that when the acquisition function is more exploitative (as opposed to exploratory), BO with MAiNGO converges in fewer iterations than with the local solvers. However, when the dataset lacks diversity, or when the acquisition function is overly exploitative, BO with MAiNGO, compared to the local solvers, is more likely to converge to a local rather than a global ly near-optimal solution of the black-box function. L-BFGS-B and multi-start mitigate this risk in BO by introducing stochasticity in the selection of the next sampling point, which enhances the exploration of uncharted regions in the search space and reduces dependence on acquisition function hyperparameters. Ultimately, suboptimal optimization of poorly chosen acquisition functions may be preferable to their optimal solution. When the acquisition function is more exploratory, BO with MAiNGO, multi-start, and L-BFGS-B achieve comparable probabilities of convergence to a globally near-optimal solution (although BO with MAiNGO may require more iterations to converge under these conditions).