mlsanityData to 7 Oct 2026

High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces

arXiv 2103.00349 ↗First posted 27 Feb 2021cs.LG stat.MLcited by 139

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) is a powerful paradigm for efficient optimization of black-box objective functions. High-dimensional BO presents a particular challenge, in part because the curse of dimensionality makes it difficult to define -- as well as do inference over -- a suitable class of surrogate models. We argue that Gaussian process surrogate models defined on sparse axis-aligned subspaces offer an attractive compromise between flexibility and parsimony. We demonstrate that our approach, which relies on Hamiltonian Monte Carlo for inference, can rapidly identify sparse subspaces relevant to modeling the unknown objective function, enabling sample-efficient high-dimensional BO. In an extensive suite of experiments comparing to existing methods for high-dimensional BO we demonstrate that our algorithm, Sparse Axis-Aligned Subspace BO (SAASBO), achieves excellent performance on several synthetic and real-world problems without the need to set problem-specific hyperparameters.

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). Builds on 19 of its 49 references that are in the map. First posted on arXiv 27 Feb 2021.

Builds on

19 of 49 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018A Tutorial on Bayesian Optimizationcited by 734
  3. 2018Pyro: Deep Universal Probabilistic Programmingcited by 379
  4. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  5. 2019Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyrocited by 187
  6. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  7. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
  8. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  9. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 103
  10. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
  11. 2020Scalable Constrained Bayesian Optimizationcited by 84
  12. 2019Combinatorial Bayesian Optimization using the Graph Cartesian Productcited by 78
7 more
  1. 2017Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Spacecited by 76
  2. 2019Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspacescited by 73
  3. 2018Bayesian Optimization of Combinatorial Structurescited by 64
  4. 2018BOCK : Bayesian Optimization with Cylindrical Kernelscited by 62
  5. 2018High Dimensional Bayesian Optimization Using Dropoutcited by 40
  6. 2013Active Learning of Linear Embeddings for Gaussian Processescited by 34
  7. 2017On the choice of the low-dimensional domain for global optimization via random embeddingscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 15; 2023: 28; 2024: 25; 2025: 37; 2026: 29.

139 papers.

Sort:

By area:

  1. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 144
  2. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 128
  3. 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
  4. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  5. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
  6. 2022Recent Advances in Bayesian Optimizationcited by 74
  7. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 70
  8. 2022MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Designcited by 66
  9. 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
  10. 2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 50
  11. 2023AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuningcited by 49
  12. 2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
38 more
  1. 2022Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimizationcited by 37
  2. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
  3. 2024Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimizationcited by 33
  4. 2024Personalized LLM Response Generation with Parameterized Memory Injectioncited by 33
  5. 2023Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spacescited by 27
  6. 2023Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddingscited by 24
  7. 2021Conditioning Sparse Variational Gaussian Processes for Online Decision-makingcited by 23
  8. 2022Query-Efficient and Scalable Black-Box Adversarial Attacks on Discrete Sequential Data via Bayesian Optimizationcited by 22
  9. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
  10. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  11. 2025Understanding High-Dimensional Bayesian Optimizationcited by 19
  12. 2023Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOBcited by 18
  13. 2021Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimizationcited by 15
  14. 2023B2Opt: Learning to Optimize Black-box Optimization with Little Budgetcited by 15
  15. 2024Bayesian optimization as a flexible and efficient design framework for sustainable process systemscited by 15
  16. 2024The Vizier Gaussian Process Bandit Algorithmcited by 14
  17. 2021Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selectioncited by 12
  18. 2022Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processescited by 11
  19. 2022Log Barriers for Safe Black-box Optimization with Application to Safe Reinforcement Learningcited by 11
  20. 2023RTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernelscited by 11
  21. 2022Automatic and effective discovery of quantum kernelscited by 10
  22. 2023A safe exploration approach to constrained Markov decision processescited by 10
  23. 2024Generative Adversarial Model-Based Optimization via Source Critic Regularizationcited by 10
  24. 2025LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithmscited by 10
  25. 2024Improving sample efficiency of high dimensional Bayesian optimization with MCMCcited by 9
  26. 2024Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimizationcited by 9
  27. 2024Hyperband-based Bayesian Optimization for Black-box Prompt Selectioncited by 9
  28. 2025We Still Don't Understand High-Dimensional Bayesian Optimizationcited by 9
  29. 2026HARBOR: Automated Harness Optimizationcited by 9
  30. 2023Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimizationcited by 8
  31. 2024Global Optimization of Gaussian Process Acquisition Functions Using a Piecewise-Linear Kernel Approximationcited by 8
  32. 2025Exploring Exploration in Bayesian Optimizationcited by 8
  33. 2022Sparse Bayesian Optimizationcited by 6
  34. 2023Additive Multi-Index Gaussian process modeling, with application to multi-physics surrogate modeling of the quark-gluon plasmacited by 6
  35. 2024Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimalcited by 6
  36. 2024EARL-BO: Reinforcement Learning for Multi-Step Lookahead, High-Dimensional Bayesian Optimizationcited by 6
  37. 2025NeST-BO: Fast Local Bayesian Optimization via Newton-Step Targeting of Gradient and Hessian Informationcited by 6
  38. 2022ODBO: Bayesian Optimization with Search Space Prescreening for Directed Protein Evolutioncited by 5

50 most cited of 139 All 139 (JSON)