mlsanityData to 14 Oct 2026

The Behavior and Convergence of Local Bayesian Optimization

arXiv 2305.15572 ↗First posted 24 May 2023cs.LG stat.MLcited by 20

Not on the map. Why?

Abstract (arXiv)

A recent development in Bayesian optimization is the use of local optimization strategies, which can deliver strong empirical performance on high-dimensional problems compared to traditional global strategies. The "folk wisdom" in the literature is that the focus on local optimization sidesteps the curse of dimensionality; however, little is known concretely about the expected behavior or convergence of Bayesian local optimization routines. We first study the behavior of the local approach, and find that the statistics of individual local solutions of Gaussian process sample paths are surprisingly good compared to what we would expect to recover from global methods. We then present the first rigorous analysis of such a Bayesian local optimization algorithm recently proposed by M"uller et al. (2021), and derive convergence rates in both the noisy and noiseless settings.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

18 of 30 references.

References · 18

  1. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  2. 2014Convex Optimization: Algorithms and Complexitycited by 575
  3. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  4. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 139
  5. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 123
  6. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 102
  7. 2020Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Searchcited by 89
  8. 2020Scalable Constrained Bayesian Optimizationcited by 84
  9. 2020On the Convergence of SGD with Biased Gradientscited by 83
  10. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 79
  11. 2020Pathwise Conditioning of Gaussian Processescited by 64
  12. 2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
6 more
  1. 2021Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spacescited by 45
  2. 2021Local policy search with Bayesian optimizationcited by 38
  3. 2022Discovering Many Diverse Solutions with Bayesian Optimizationcited by 31
  4. 2022Local Bayesian optimization via maximizing probability of descentcited by 30
  5. 2018Tight Regret Bounds for Bayesian Optimization in One Dimensioncited by 20
  6. 2016Bayesian Optimization with Exponential Convergencecited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 7; 2026: 6.

20 papers.

Sort:

  1. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 92
  2. 2025We Still Don't Understand High-Dimensional Bayesian Optimizationcited by 12
  3. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 10
  4. 2024Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimizationcited by 9
  5. 2025NeST-BO: Fast Local Bayesian Optimization via Newton-Step Targeting of Gradient and Hessian Informationcited by 6
  6. 2026BayeSQP: Bayesian Optimization through Sequential Quadratic Programmingcited by 5
  7. 2024CAGES: Cost-Aware Gradient Entropy Search for Efficient Local Multi-Fidelity Bayesian Optimizationcited by 4
  8. 2024Simulation-Aided Policy Tuning for Black-Box Robot Learningcited by 4
  9. 2024Safe Bayesian Optimization for the Control of High-Dimensional Embodied Systemscited by 4
  10. 2024Local Bayesian Optimization for Controller Tuning with Crash Constraintscited by 3
  11. 2025Deterministic Global Optimization of the Acquisition Function in Bayesian Optimization: To Do or Not To Do?cited by 3
  12. 2025End-to-End Optimal Detector Design with Mutual Information Surrogatescited by 2
8 more
  1. 2025Differentially Private Hyperparameter Tuning using Local Bayesian Optimizationcited by 1
  2. 2025Local Entropy Search over Descent Sequences for Bayesian Optimizationcited by 1
  3. 2026Adaptive Candidate Point Thompson Sampling for High-Dimensional Bayesian Optimizationcited by 1
  4. 2025Adaptive Replication Strategies in Trust-Region-Based Bayesian Optimization of Stochastic Functionscited by 0
  5. 2026Local Constrained Bayesian Optimizationcited by 0
  6. 2026Local Preferential Bayesian Optimizationcited by 0
  7. 2026GRAPE: Gradient Refinement and Progress-Aware Exploitation for Query-Efficient High-Dimensional Bayesian Optimizationcited by 0
  8. 2026Isotropic Gaussian Processes Improve Vanilla Bayesian Optimization in High Dimensionscited by 0