mlsanityData to 7 Oct 2026

High Dimensional Bayesian Optimization using Lasso Variable Selection

arXiv 2504.01743 ↗First posted 2 Apr 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) is a leading method for optimizing expensive black-box optimization and has been successfully applied across various scenarios. However, BO suffers from the curse of dimensionality, making it challenging to scale to high-dimensional problems. Existing work has adopted a variable selection strategy to select and optimize only a subset of variables iteratively. Although this approach can mitigate the high-dimensional challenge in BO, it still leads to sample inefficiency. To address this issue, we introduce a novel method that identifies important variables by estimating the length scales of Gaussian process kernels. Next, we construct an effective search region consisting of multiple subspaces and optimize the acquisition function within this region, focusing on only the important variables. We demonstrate that our proposed method achieves cumulative regret with a sublinear growth rate in the worst case while maintaining computational efficiency. Experiments on high-dimensional synthetic functions and real-world problems show that our method achieves state-of-the-art performance.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 36 of its 56 references that are in the map. First posted on arXiv 2 Apr 2025.

Builds on

36 of 56 references.

References · 36

  1. 2018A Tutorial on Bayesian Optimizationcited by 734
  2. 2016The CMA Evolution Strategy: A Tutorialcited by 364
  3. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  4. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  5. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 144
  6. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  7. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
  8. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 131
  9. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  10. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
  11. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  12. 2021A survey on high-dimensional Gaussian process modeling with application to Bayesian optimizationcited by 76
24 more
  1. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  2. 2022πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimizationcited by 69
  3. 2018High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groupscited by 58
  4. 2021Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spacescited by 51
  5. 2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
  6. 2018High Dimensional Bayesian Optimization Using Dropoutcited by 40
  7. 2022Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimizationcited by 37
  8. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
  9. 2017Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimizationcited by 33
  10. 2024Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimizationcited by 33
  11. 2018Celer: a Fast Solver for the Lasso with Dual Extrapolationcited by 31
  12. 2020Optimal Transport Kernels for Sequential and Parallel Neural Architecture Searchcited by 29
  13. 2020High-Dimensional Bayesian Optimization via Tree-Structured Additive Modelscited by 22
  14. 2019Knowing The What But Not The Where in Bayesian Optimizationcited by 20
  15. 2022Additive Gaussian Processes Revisitedcited by 19
  16. 2023Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOBcited by 18
  17. 2022Distributionally Robust Bayesian Optimization with varphi-divergencescited by 16
  18. 2022Regret Bounds for Expected Improvement Algorithms in Gaussian Process Bandit Optimizationcited by 14
  19. 2021Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selectioncited by 12
  20. 2019Trading Convergence Rate with Computational Budget in High Dimensional Bayesian Optimizationcited by 7
  21. 2021Regret Bounds for Gaussian-Process Optimization in Large Domainscited by 7
  22. 2020Sub-linear Regret Bounds for Bayesian Optimisation in Unknown Search Spacescited by 5
  23. 2023Provably Efficient Bayesian Optimization with Unknown Gaussian Process Hyperparameter Estimationcited by 2
  24. 2024High-Dimensional Bayesian Optimization via Random Projection of Manifold Subspacescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?cited by 2
  2. 2026Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisationcited by 0
  3. 2026Agentic High-Dimensional Bayesian Optimization with Hypothesis- and Evidence-Guided Searchcited by 0