Enhancing Trust-Region Bayesian Optimization via Newton Methods
Not on the map. Why?
Abstract (arXiv)
Bayesian Optimization (BO) has been widely applied to optimize expensive black-box functions while retaining sample efficiency. However, scaling BO to high-dimensional spaces remains challenging. Existing literature proposes performing standard BO in multiple local trust regions (TuRBO) for heterogeneous modeling of the objective function and avoiding over-exploration. Despite its advantages, using local Gaussian Processes (GPs) reduces sampling efficiency compared to a global GP. To enhance sampling efficiency while preserving heterogeneous modeling, we propose to construct multiple local quadratic models using gradients and Hessians from a global GP, and select new sample points by solving the bound-constrained quadratic program. Additionally, we address the issue of vanishing gradients of GPs in high-dimensional spaces. We provide a convergence analysis and demonstrate through experimental results that our method enhances the efficacy of TuRBO and outperforms a wide range of high-dimensional BO techniques on synthetic functions and real-world applications.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 29 of its 40 references that are in the map. First posted on arXiv 25 Aug 2025.
Builds on
29 of 40 references.
References · 29
- 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
- 2016The CMA Evolution Strategy: A Tutorialcited by 377
- 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
- 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 204
- 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 182
- 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 141
- 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 139
- 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 117
- 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 92
- 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 92
- 2020Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Searchcited by 89
- 2019Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspacescited by 73
17 more
- 2019Uniform Error Bounds for Gaussian Process Regression with Application to Safe Controlcited by 64
- 2018High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groupscited by 59
- 2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 52
- 2017Batched High-dimensional Bayesian Optimization via Structural Kernel Learningcited by 51
- 2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
- 2018High Dimensional Bayesian Optimization Using Dropoutcited by 40
- 2021Local policy search with Bayesian optimizationcited by 38
- 2022Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimizationcited by 37
- 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 36
- 2013Active Learning of Linear Embeddings for Gaussian Processescited by 34
- 2015Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functionscited by 31
- 2022Local Bayesian optimization via maximizing probability of descentcited by 30
- 2017On the choice of the low-dimensional domain for global optimization via random embeddingscited by 25
- 2020High-Dimensional Bayesian Optimization via Tree-Structured Additive Modelscited by 22
- 2019High Dimensional Bayesian Optimization via Supervised Dimension Reductioncited by 21
- 2021Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selectioncited by 12
- 2023Unleashing the Potential of Acquisition Functions in High-Dimensional Bayesian Optimizationcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.