mlsanityData to 7 Oct 2026

Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection

arXiv 2109.09264 ↗First posted 20 Sep 2021cs.LG stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

Bayesian Optimization (BO) is a method for globally optimizing black-box functions. While BO has been successfully applied to many scenarios, developing effective BO algorithms that scale to functions with high-dimensional domains is still a challenge. Optimizing such functions by vanilla BO is extremely time-consuming. Alternative strategies for high-dimensional BO that are based on the idea of embedding the high-dimensional space to the one with low dimension are sensitive to the choice of the embedding dimension, which needs to be pre-specified. We develop a new computationally efficient high-dimensional BO method that exploits variable selection. Our method is able to automatically learn axis-aligned sub-spaces, i.e. spaces containing selected variables, without the demand of any pre-specified hyperparameters. We theoretically analyze the computational complexity of our algorithm and derive the regret bound. We empirically show the efficacy of our method on several synthetic and real problems.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 24 of its 71 references that are in the map. First posted on arXiv 20 Sep 2021.

Builds on

24 of 71 references.

References · 24

  1. 2018A Tutorial on Bayesian Optimizationcited by 734
  2. 2016The CMA Evolution Strategy: A Tutorialcited by 364
  3. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  4. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  5. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  6. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
  7. 2016Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Roboticscited by 134
  8. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 131
  9. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
  10. 2019Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspacescited by 73
  11. 2019Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonflycited by 66
  12. 2018High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groupscited by 58
12 more
  1. 2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
  2. 2017Batched High-dimensional Bayesian Optimization via Structural Kernel Learningcited by 51
  3. 2021Improving black-box optimization in VAE latent space using decoder uncertaintycited by 51
  4. 2017Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methodscited by 49
  5. 2017Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimizationcited by 33
  6. 2015Bayesian Optimization for Synthetic Gene Designcited by 21
  7. 2017Constrained Bayesian Optimization for Automatic Chemical Designcited by 20
  8. 2017Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distributioncited by 15
  9. 2015Bayesian Optimization with Dimension Scheduling: Application to Biological Systemscited by 9
  10. 2019Trading Convergence Rate with Computational Budget in High Dimensional Bayesian Optimizationcited by 7
  11. 2014Automated Machine Learning on Big Data using Stochastic Algorithm Tuningcited by 4
  12. 2016Minimum Regret Search for Single- and Multi-Task Optimizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 5; 2026: 4.

12 papers.

  1. 2022Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimizationcited by 37
  2. 2025High Dimensional Bayesian Optimization using Lasso Variable Selectioncited by 3
  3. 2024Navigating in High-Dimensional Search Space: A Hierarchical Bayesian Optimization Approachcited by 2
  4. 2025Enhancing Trust-Region Bayesian Optimization via Newton Methodscited by 2
  5. 2025Leveraging Axis-Aligned Subspaces for High-Dimensional Bayesian Optimization with Group Testingcited by 1
  6. 2025Bencher: Simple and Reproducible Benchmarking for Black-Box Optimizationcited by 1
  7. 2026Extending Multi-Source Bayesian Optimization With Causality Principlescited by 1
  8. 2023Learning Relevant Contextual Variables Within Bayesian Optimizationcited by 0
  9. 2025Efficient Sliced Wasserstein Distance Computation via Adaptive Bayesian Optimizationcited by 0
  10. 2026Dynamic Hyperparameter Importance for Efficient Multi-Objective Optimizationcited by 0
  11. 2026BONSAI: Bayesian Optimization with Natural Simplicity and Interpretabilitycited by 0
  12. 2026Goal-Oriented Influence-Maximizing Data Acquisition for Learning and Optimizationcited by 0