mlsanityData to 14 Oct 2026

An Adaptive Dimension Reduction Estimation Method for High-dimensional Bayesian Optimization

arXiv 2403.05425 ↗First posted 8 Mar 2024stat.ML stat.MEcited by 1

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) has shown impressive results in a variety of applications within low-to-moderate dimensional Euclidean spaces. However, extending BO to high-dimensional settings remains a significant challenge. We address this challenge by proposing a two-step optimization framework. Initially, we identify the effective dimension reduction (EDR) subspace for the objective function using the minimum average variance estimation (MAVE) method. Subsequently, we construct a Gaussian process model within this EDR subspace and optimize it using the expected improvement criterion. Our algorithm offers the flexibility to operate these steps either concurrently or in sequence. In the sequential approach, we meticulously balance the exploration-exploitation trade-off by distributing the sampling budget between subspace estimation and function optimization, and the convergence rate of our algorithm in high-dimensional contexts has been established. Numerical experiments validate the efficacy of our method in challenging scenarios.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 62 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

23 of 62 references.

References · 23

  1. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  2. 2017On Kernelized Multi-armed Banditscited by 352
  3. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 206
  4. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 204
  5. 2018Gaussian Processes and Kernel Methods: A Review on Connections and Equivalencescited by 192
  6. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 182
  7. 2013Finite-Time Analysis of Kernelised Contextual Banditscited by 177
  8. 2020On Information Gain and Regret Bounds in Gaussian Process Banditscited by 158
  9. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 139
  10. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 104
  11. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 92
  12. 2015Bayesian optimization for materials designcited by 80
11 more
  1. 2017Lower Bounds on Regret for Noisy Gaussian Process Bandit Optimizationcited by 78
  2. 2021A survey on high-dimensional Gaussian process modeling with application to Bayesian optimizationcited by 77
  3. 2016The Parallel Knowledge Gradient Method for Batch Bayesian Optimizationcited by 74
  4. 2016Parallel Bayesian Global Optimization of Expensive Functionscited by 44
  5. 2018High Dimensional Bayesian Optimization Using Dropoutcited by 40
  6. 2014Theoretical Analysis of Bayesian Optimisation with Unknown Gaussian Process Hyper-Parameterscited by 31
  7. 2018Regret bounds for meta Bayesian optimization with an unknown Gaussian process priorcited by 31
  8. 2017Decentralized High-Dimensional Bayesian Optimization with Factor Graphscited by 23
  9. 2019High Dimensional Bayesian Optimization via Supervised Dimension Reductioncited by 21
  10. 2016Sparse Sliced Inverse Regression Via Lassocited by 8
  11. 2022Adjusted Expected Improvement for Cumulative Regret Minimization in Noisy Bayesian Optimizationcited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 23