mlsanityData to 7 Oct 2026

Scalable Global Optimization via Local Bayesian Optimization

arXiv 1910.01739 ↗First posted 3 Oct 2019cs.LG stat.MLcited by 359

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization has recently emerged as a popular method for the sample-efficient optimization of expensive black-box functions. However, the application to high-dimensional problems with several thousand observations remains challenging, and on difficult problems Bayesian optimization is often not competitive with other paradigms. In this paper we take the view that this is due to the implicit homogeneity of the global probabilistic models and an overemphasized exploration that results from global acquisition. This motivates the design of a local probabilistic approach for global optimization of large-scale high-dimensional problems. We propose the textttTuRBO algorithm that fits a collection of local models and performs a principled global allocation of samples across these models via an implicit bandit approach. A comprehensive evaluation demonstrates that textttTuRBO outperforms state-of-the-art methods from machine learning and operations research on problems spanning reinforcement learning, robotics, and the natural sciences.

Key facts. Cited by 359 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 43; 2022: 41; 2023: 55; 2024: 70; 2025: 74; 2026: 76.

359 papers.

Sort:

By area:

  1. 2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 229
  2. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  3. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 144
  4. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 143
  5. 2023Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performancecited by 143
  6. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  7. 2024Large Language Models to Enhance Bayesian Optimizationcited by 127
  8. 2021HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPOcited by 111
  9. 2021DEUP: Direct Epistemic Uncertainty Predictioncited by 102
  10. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 99
  11. 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
  12. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
38 more
  1. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
  2. 2023DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Trainingcited by 76
  3. 2022Recent Advances in Bayesian Optimizationcited by 74
  4. 2022Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakagecited by 72
  5. 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
  6. 2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
  7. 2021Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spacescited by 51
  8. 2022Pareto Set Learning for Expensive Multi-Objective Optimizationcited by 51
  9. 2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 50
  10. 2023A Study of Bayesian Neural Network Surrogates for Bayesian Optimizationcited by 50
  11. 2021Benchmarking the Performance of Bayesian Optimization across Multiple Experimental Materials Science Domainscited by 46
  12. 2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
  13. 2021Local policy search with Bayesian optimizationcited by 37
  14. 2022Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimizationcited by 37
  15. 2022Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterizationcited by 35
  16. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
  17. 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Surveycited by 34
  18. 2021Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Informationcited by 33
  19. 2024Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimizationcited by 33
  20. 2021Differentially Private Federated Bayesian Optimization with Distributed Explorationcited by 32
  21. 2023Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisationcited by 32
  22. 2024Model-Based Diffusion for Trajectory Optimizationcited by 30
  23. 2022Discovering Many Diverse Solutions with Bayesian Optimizationcited by 29
  24. 2022Local Bayesian optimization via maximizing probability of descentcited by 29
  25. 2023Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformerscited by 29
  26. 2021TREGO: a Trust-Region Framework for Efficient Global Optimizationcited by 28
  27. 2022Bayesian Optimization with Conformal Prediction Setscited by 27
  28. 2023Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spacescited by 27
  29. 2022Sample-Then-Optimize Batch Neural Thompson Samplingcited by 25
  30. 2023Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddingscited by 24
  31. 2021Conditioning Sparse Variational Gaussian Processes for Online Decision-makingcited by 23
  32. 2023Macro Placement by Wire-Mask-Guided Black-Box Optimizationcited by 23
  33. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
  34. 2023A General Framework for User-Guided Bayesian Optimizationcited by 22
  35. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  36. 2024LEDRO: LLM-Enhanced Design Space Reduction and Optimization for Analog Circuitscited by 22
  37. 2025Completed Hyperparameter Transfer across Modules, Width, Depth, Batch and Durationcited by 22
  38. 2021Automatic Termination for Hyperparameter Optimizationcited by 21

50 most cited of 359 All 359 (JSON)