# Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection

arXiv: https://arxiv.org/abs/2109.09264 · HTML: https://mlsanity.com/paper/2109.09264/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 71 references that are in the map.
- First posted on arXiv 20 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Bayesian Optimization in a Billion Dimensions via Random Embeddings](https://mlsanity.com/paper/1301.1942/) (2013)
- [High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces](https://mlsanity.com/paper/2103.00349/) (2021)
- [High Dimensional Bayesian Optimisation and Bandits via Additive Models](https://mlsanity.com/paper/1503.01673/) (2015)
- [Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics](https://mlsanity.com/paper/1602.04450/) (2016)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2001.11659/) (2020)
- [Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces](https://mlsanity.com/paper/1902.03229/) (2019)
- [Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly](https://mlsanity.com/paper/1903.06694/) (2019)
- [High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups](https://mlsanity.com/paper/1802.07028/) (2018)
- [High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning](https://mlsanity.com/paper/2106.03609/) (2021)
- [Batched High-dimensional Bayesian Optimization via Structural Kernel Learning](https://mlsanity.com/paper/1703.01973/) (2017)
- [Improving black-box optimization in VAE latent space using decoder uncertainty](https://mlsanity.com/paper/2107.00096/) (2021)
- [Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods](https://mlsanity.com/paper/1712.09677/) (2017)
- [Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization](https://mlsanity.com/paper/1703.01250/) (2017)
- [Bayesian Optimization for Synthetic Gene Design](https://mlsanity.com/paper/1505.01627/) (2015)
- [Constrained Bayesian Optimization for Automatic Chemical Design](https://mlsanity.com/paper/1709.05501/) (2017)
- [Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution](https://mlsanity.com/paper/1712.08048/) (2017)
- [Bayesian Optimization with Dimension Scheduling: Application to Biological Systems](https://mlsanity.com/paper/1511.05385/) (2015)
- [Trading Convergence Rate with Computational Budget in High Dimensional Bayesian Optimization](https://mlsanity.com/paper/1911.11950/) (2019)
- [Automated Machine Learning on Big Data using Stochastic Algorithm Tuning](https://mlsanity.com/paper/1407.7969/) (2014)
- [Minimum Regret Search for Single- and Multi-Task Optimization](https://mlsanity.com/paper/1602.01064/) (2016)

## Built on by (most cited first, 12 of 12)

- [Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization](https://mlsanity.com/paper/2210.01628/) (2022)
- [High Dimensional Bayesian Optimization using Lasso Variable Selection](https://mlsanity.com/paper/2504.01743/) (2025)
- [Navigating in High-Dimensional Search Space: A Hierarchical Bayesian Optimization Approach](https://mlsanity.com/paper/2410.23148/) (2024)
- [Leveraging Axis-Aligned Subspaces for High-Dimensional Bayesian Optimization with Group Testing](https://mlsanity.com/paper/2504.06111/) (2025)
- [Bencher: Simple and Reproducible Benchmarking for Black-Box Optimization](https://mlsanity.com/paper/2505.21321/) (2025)
- [Enhancing Trust-Region Bayesian Optimization via Newton Methods](https://mlsanity.com/paper/2508.18423/) (2025)
- [Extending Multi-Source Bayesian Optimization With Causality Principles](https://mlsanity.com/paper/2602.14791/) (2026)
- [Learning Relevant Contextual Variables Within Bayesian Optimization](https://mlsanity.com/paper/2305.14120/) (2023)
- [Efficient Sliced Wasserstein Distance Computation via Adaptive Bayesian Optimization](https://mlsanity.com/paper/2509.17405/) (2025)
- [Dynamic Hyperparameter Importance for Efficient Multi-Objective Optimization](https://mlsanity.com/paper/2601.03166/) (2026)
- [BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability](https://mlsanity.com/paper/2602.07144/) (2026)
- [Goal-Oriented Influence-Maximizing Data Acquisition for Learning and Optimization](https://mlsanity.com/paper/2602.19578/) (2026)

## Abstract (arXiv)

Bayesian Optimization (BO) is a method for globally optimizing black-box functions. While BO has been successfully applied to many scenarios, developing effective BO algorithms that scale to functions with high-dimensional domains is still a challenge. Optimizing such functions by vanilla BO is extremely time-consuming. Alternative strategies for high-dimensional BO that are based on the idea of embedding the high-dimensional space to the one with low dimension are sensitive to the choice of the embedding dimension, which needs to be pre-specified. We develop a new computationally efficient high-dimensional BO method that exploits variable selection. Our method is able to automatically learn axis-aligned sub-spaces, i.e. spaces containing selected variables, without the demand of any pre-specified hyperparameters. We theoretically analyze the computational complexity of our algorithm and derive the regret bound. We empirically show the efficacy of our method on several synthetic and real problems.