mlsanityData to 7 Oct 2026

We Still Don't Understand High-Dimensional Bayesian Optimization

arXiv 2512.00170 ↗First posted 28 Nov 2025cs.LG stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Existing high-dimensional Bayesian optimization (BO) methods aim to overcome the curse of dimensionality by carefully encoding structural assumptions, from locality to sparsity to smoothness, into the optimization procedure. Surprisingly, we demonstrate that these approaches are outperformed by arguably the simplest method imaginable: Bayesian linear regression. After applying a geometric transformation to avoid boundary-seeking behavior, Gaussian processes with linear kernels match state-of-the-art performance on tasks with 60- to 6,000-dimensional search spaces. Linear models offer numerous advantages over their non-parametric counterparts: they afford closed-form sampling and their computation scales linearly with data, a fact we exploit on molecular optimization tasks with >20,000 observations. Coupled with empirical analyses, our results suggest the need to depart from past intuitions about BO methods in high-dimensions.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 38 of its 60 references that are in the map. First posted on arXiv 28 Nov 2025.

Builds on

38 of 60 references.

References · 38

  1. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  2. 2013Gaussian Processes for Big Datacited by 377
  3. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  4. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 294
  5. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  6. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 144
  7. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  8. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
  9. 2020Efficiently Sampling Functions from Gaussian Process Posteriorscited by 125
  10. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  11. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
  12. 2020Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Searchcited by 87
26 more
  1. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  2. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
  3. 2017Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Spacecited by 76
  4. 2019Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspacescited by 73
  5. 2018BOCK : Bayesian Optimization with Cylindrical Kernelscited by 62
  6. 2018High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groupscited by 58
  7. 2021Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spacescited by 51
  8. 2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 50
  9. 2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
  10. 2018High Dimensional Bayesian Optimization Using Dropoutcited by 40
  11. 2020Scalable Thompson Sampling using Sparse Gaussian Process Modelscited by 39
  12. 2021Local policy search with Bayesian optimizationcited by 37
  13. 2022Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimizationcited by 37
  14. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
  15. 2024Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimizationcited by 33
  16. 2023Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisationcited by 32
  17. 2022Local Bayesian optimization via maximizing probability of descentcited by 29
  18. 2023Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spacescited by 27
  19. 2020High-Dimensional Bayesian Optimization via Tree-Structured Additive Modelscited by 22
  20. 2023The Behavior and Convergence of Local Bayesian Optimizationcited by 19
  21. 2025Understanding High-Dimensional Bayesian Optimizationcited by 19
  22. 2023Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOBcited by 18
  23. 2024Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimizationcited by 9
  24. 2024Approximation-Aware Bayesian Optimizationcited by 8
  25. 2025Exploring Exploration in Bayesian Optimizationcited by 8
  26. 2025Leveraging Axis-Aligned Subspaces for High-Dimensional Bayesian Optimization with Group Testingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimizationcited by 1
  2. 2026Search at the Cost of Sampling: Nearly-Instant Latent Space Bayesian Optimizationcited by 1
  3. 2026Weighted Wasserstein Barycenter of Gaussian Processes for exotic Bayesian Optimization taskscited by 0
  4. 2026Gradient-Informed Bayesian and Interior Point Optimization for Efficient Inverse Design in Nanophotonicscited by 0
  5. 2026Rethinking Trust Region Bayesian Optimization in High Dimensionscited by 0
  6. 2026GRAPE: Gradient Refinement and Progress-Aware Exploitation for Query-Efficient High-Dimensional Bayesian Optimizationcited by 0
  7. 2026Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisationcited by 0
  8. 2026Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methodscited by 0
  9. 2026Bayesian Optimisation under State-Preservation Constraintscited by 0