mlsanityData to 14 Oct 2026

Isotropic Gaussian Processes Improve Vanilla Bayesian Optimization in High Dimensions

arXiv 2610.05780 ↗First posted 5 Oct 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

High-dimensional Bayesian optimization (BO) often fits Gaussian process (GP) surrogates from far fewer observations than input dimensions. Modern Vanilla BO can perform well in this regime with dimension-aware priors, initialization, and acquisition optimization, but it typically retains automatic relevance determination (ARD), fitting one lengthscale per input coordinate. We study this modeling choice and propose Iso-BO, a controlled modification that replaces the ARD GP with an isotropic GP using one shared lengthscale while keeping the surrounding BO pipeline matched. For radial kernels, we show that the marginal log likelihood (MLL) depends on the inverse-squared ARD lengthscales only through weighted pairwise distances among the observed inputs. The current design can therefore leave some ARD directions exactly invisible or only weakly constrained by the MLL. Iso-BO removes coordinatewise reweighting and fits a single shared scale instead. Lengthscale-fitting and predictive-density diagnostics show that this finite-data effect appears in practice, including when the data-generating process is anisotropic. Across GP-prior, synthetic, and real-world benchmarks, Iso-BO often improves over matched modern Vanilla BO and remains competitive with the included high-dimensional BO baselines under the tested budgets. Stress tests also show the expected boundary wherein sufficiently strong, learnable anisotropy can favor the more flexible ARD model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

30 of 44 references.

References · 30

  1. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 609
  2. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  3. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  4. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 182
  5. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 149
  6. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 141
  7. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 139
  8. 2020Efficiently Sampling Functions from Gaussian Process Posteriorscited by 127
  9. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 117
  10. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 92
  11. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 92
  12. 2020Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Searchcited by 89
18 more
  1. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 79
  2. 2021A survey on high-dimensional Gaussian process modeling with application to Bayesian optimizationcited by 77
  3. 2022Recent Advances in Bayesian Optimizationcited by 76
  4. 2018BOCK : Bayesian Optimization with Cylindrical Kernelscited by 65
  5. 2018High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groupscited by 59
  6. 2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 52
  7. 2021Local policy search with Bayesian optimizationcited by 38
  8. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 36
  9. 2024Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimizationcited by 34
  10. 2022Local Bayesian optimization via maximizing probability of descentcited by 30
  11. 2023The Behavior and Convergence of Local Bayesian Optimizationcited by 20
  12. 2025Understanding High-Dimensional Bayesian Optimizationcited by 20
  13. 2025We Still Don't Understand High-Dimensional Bayesian Optimizationcited by 12
  14. 2025Exploring Exploration in Bayesian Optimizationcited by 9
  15. 2024High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategycited by 6
  16. 2025High Dimensional Bayesian Optimization using Lasso Variable Selectioncited by 4
  17. 2025Leveraging Axis-Aligned Subspaces for High-Dimensional Bayesian Optimization with Group Testingcited by 2
  18. 2026Rethinking Trust Region Bayesian Optimization in High Dimensionscited by 2

Built on by

Not cited yet.