# The Behavior and Convergence of Local Bayesian Optimization

arXiv: https://arxiv.org/abs/2305.15572 · HTML: https://mlsanity.com/paper/2305.15572/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 18 of 18)

- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Convex Optimization: Algorithms and Complexity](https://mlsanity.com/paper/1405.4980/) (2014)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [High Dimensional Bayesian Optimisation and Bandits via Additive Models](https://mlsanity.com/paper/1503.01673/) (2015)
- [Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining](https://mlsanity.com/paper/2006.09191/) (2020)
- [Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders](https://mlsanity.com/paper/2203.12742/) (2022)
- [Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search](https://mlsanity.com/paper/2007.00708/) (2020)
- [Scalable Constrained Bayesian Optimization](https://mlsanity.com/paper/2002.08526/) (2020)
- [On the Convergence of SGD with Biased Gradients](https://mlsanity.com/paper/2008.00051/) (2020)
- [Local Latent Space Bayesian Optimization over Structured Inputs](https://mlsanity.com/paper/2201.11872/) (2022)
- [Pathwise Conditioning of Gaussian Processes](https://mlsanity.com/paper/2011.04026/) (2020)
- [High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning](https://mlsanity.com/paper/2106.03609/) (2021)
- [Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces](https://mlsanity.com/paper/2111.01186/) (2021)
- [Local policy search with Bayesian optimization](https://mlsanity.com/paper/2106.11899/) (2021)
- [Discovering Many Diverse Solutions with Bayesian Optimization](https://mlsanity.com/paper/2210.10953/) (2022)
- [Local Bayesian optimization via maximizing probability of descent](https://mlsanity.com/paper/2210.11662/) (2022)
- [Tight Regret Bounds for Bayesian Optimization in One Dimension](https://mlsanity.com/paper/1805.11792/) (2018)
- [Bayesian Optimization with Exponential Convergence](https://mlsanity.com/paper/1604.01348/) (2016)

## Built on by (most cited first, 20 of 20)

- [Vanilla Bayesian Optimization Performs Great in High Dimensions](https://mlsanity.com/paper/2402.02229/) (2024)
- [We Still Don't Understand High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2512.00170/) (2025)
- [Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Tasks](https://mlsanity.com/paper/2410.22296/) (2024)
- [Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimization](https://mlsanity.com/paper/2405.15285/) (2024)
- [NeST-BO: Fast Local Bayesian Optimization via Newton-Step Targeting of Gradient and Hessian Information](https://mlsanity.com/paper/2510.05516/) (2025)
- [BayeSQP: Bayesian Optimization through Sequential Quadratic Programming](https://mlsanity.com/paper/2602.03232/) (2026)
- [CAGES: Cost-Aware Gradient Entropy Search for Efficient Local Multi-Fidelity Bayesian Optimization](https://mlsanity.com/paper/2405.07760/) (2024)
- [Simulation-Aided Policy Tuning for Black-Box Robot Learning](https://mlsanity.com/paper/2411.14246/) (2024)
- [Safe Bayesian Optimization for the Control of High-Dimensional Embodied Systems](https://mlsanity.com/paper/2412.20350/) (2024)
- [Local Bayesian Optimization for Controller Tuning with Crash Constraints](https://mlsanity.com/paper/2411.16267/) (2024)
- [Deterministic Global Optimization of the Acquisition Function in Bayesian Optimization: To Do or Not To Do?](https://mlsanity.com/paper/2503.03625/) (2025)
- [End-to-End Optimal Detector Design with Mutual Information Surrogates](https://mlsanity.com/paper/2503.14342/) (2025)
- [Differentially Private Hyperparameter Tuning using Local Bayesian Optimization](https://mlsanity.com/paper/2502.06044/) (2025)
- [Local Entropy Search over Descent Sequences for Bayesian Optimization](https://mlsanity.com/paper/2511.19241/) (2025)
- [Adaptive Candidate Point Thompson Sampling for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2604.08891/) (2026)
- [Adaptive Replication Strategies in Trust-Region-Based Bayesian Optimization of Stochastic Functions](https://mlsanity.com/paper/2504.20527/) (2025)
- [Local Constrained Bayesian Optimization](https://mlsanity.com/paper/2603.07965/) (2026)
- [Local Preferential Bayesian Optimization](https://mlsanity.com/paper/2606.02351/) (2026)
- [GRAPE: Gradient Refinement and Progress-Aware Exploitation for Query-Efficient High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2608.25116/) (2026)
- [Isotropic Gaussian Processes Improve Vanilla Bayesian Optimization in High Dimensions](https://mlsanity.com/paper/2610.05780/) (2026)

## Abstract (arXiv)

A recent development in Bayesian optimization is the use of local optimization strategies, which can deliver strong empirical performance on high-dimensional problems compared to traditional global strategies. The "folk wisdom" in the literature is that the focus on local optimization sidesteps the curse of dimensionality; however, little is known concretely about the expected behavior or convergence of Bayesian local optimization routines. We first study the behavior of the local approach, and find that the statistics of individual local solutions of Gaussian process sample paths are surprisingly good compared to what we would expect to recover from global methods. We then present the first rigorous analysis of such a Bayesian local optimization algorithm recently proposed by M"uller et al. (2021), and derive convergence rates in both the noisy and noiseless settings.