mlsanityData to 7 Oct 2026

High Dimensional Bayesian Optimisation and Bandits via Additive Models

arXiv 1503.01673 ↗First posted 5 Mar 2015stat.ML cs.LGcited by 137

Not on the map. Why?

Abstract (arXiv)

Bayesian Optimisation (BO) is a technique used in optimising a D-dimensional function which is typically expensive to evaluate. While there have been many successes for BO in low dimensions, scaling it to high dimensions has been notoriously difficult. Existing literature on the topic are under very restrictive settings. In this paper, we identify two key challenges in this endeavour. We tackle these challenges by assuming an additive structure for the function. This setting is substantially more expressive and contains a richer class of functions than previous work. We prove that, for additive functions the regret has only linear dependence on D even though the function depends on all D dimensions. We also demonstrate several other statistical and computational benefits in our framework. Via synthetic examples, a scientific simulation and a face detection problem we demonstrate that our method outperforms naive BO on additive functions and on several examples where the function is not additive.

Key facts. Cited by 137 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Mar 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 18; 2023: 25; 2024: 25; 2025: 26; 2026: 18.

137 papers.

Sort:

By area:

  1. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 143
  2. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  3. 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
  4. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  5. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
  6. 2022Recent Advances in Bayesian Optimizationcited by 74
  7. 2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
  8. 2022BBTv2: Towards a Gradient-Free Future with Large Language Modelscited by 53
  9. 2021Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spacescited by 51
  10. 2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 50
  11. 2023A Study of Bayesian Neural Network Surrogates for Bayesian Optimizationcited by 50
  12. 2021Practical and Rigorous Uncertainty Bounds for Gaussian Process Regressioncited by 49
38 more
  1. 2022Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimizationcited by 37
  2. 2022Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterizationcited by 35
  3. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
  4. 2024Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimizationcited by 33
  5. 2022Discovering Many Diverse Solutions with Bayesian Optimizationcited by 29
  6. 2021TREGO: a Trust-Region Framework for Efficient Global Optimizationcited by 28
  7. 2022Sample-Then-Optimize Batch Neural Thompson Samplingcited by 25
  8. 2023Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddingscited by 24
  9. 2022Query-Efficient and Scalable Black-Box Adversarial Attacks on Discrete Sequential Data via Bayesian Optimizationcited by 22
  10. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
  11. 2023The Behavior and Convergence of Local Bayesian Optimizationcited by 19
  12. 2025Understanding High-Dimensional Bayesian Optimizationcited by 19
  13. 2023Active Learning and Bayesian Optimization: a Unified Perspective to Learn with a Goalcited by 17
  14. 2023Randomized Gaussian Process Upper Confidence Bound with Tighter Bayesian Regret Boundscited by 16
  15. 2024Principled Preferential Bayesian Optimizationcited by 16
  16. 2021Derivative-Free Reinforcement Learning: A Reviewcited by 15
  17. 2023B2Opt: Learning to Optimize Black-box Optimization with Little Budgetcited by 15
  18. 2023Posterior Sampling-Based Bayesian Optimization with Tighter Bayesian Regret Boundscited by 15
  19. 2025Improved Regret Bounds for Gaussian Process Upper Confidence Bound in Bayesian Optimizationcited by 13
  20. 2021Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selectioncited by 12
  21. 2025Posterior Inference with Diffusion Models for High-dimensional Black-box Optimizationcited by 12
  22. 2021Bias-Robust Bayesian Optimization via Dueling Banditscited by 11
  23. 2022Parallel MCMC Without Embarrassing Failurescited by 11
  24. 2022Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processescited by 11
  25. 2024Functional Graphical Models: Structure Enables Offline Data-Driven Optimizationcited by 11
  26. 2021Revisiting Bayesian Optimization in the light of the COCO benchmarkcited by 10
  27. 2022Graph Neural Network Banditscited by 10
  28. 2022Benchmarking Active Learning Strategies for Materials Optimization and Discoverycited by 9
  29. 2023PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidancecited by 9
  30. 2024Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimizationcited by 9
  31. 2024Hyperband-based Bayesian Optimization for Black-box Prompt Selectioncited by 9
  32. 2025We Still Don't Understand High-Dimensional Bayesian Optimizationcited by 9
  33. 2023Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimizationcited by 8
  34. 2024"Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihoodcited by 8
  35. 2024Global Optimization of Gaussian Process Acquisition Functions Using a Piecewise-Linear Kernel Approximationcited by 8
  36. 2021Stochastic Gradient Line Bayesian Optimization for Efficient Noise-Robust Optimization of Parameterized Quantum Circuitscited by 7
  37. 2021Incremental Search Space Construction for Machine Learning Pipeline Synthesiscited by 6
  38. 2023Efficient Planning in Combinatorial Action Spaces with Applications to Cooperative Multi-Agent Reinforcement Learningcited by 6

50 most cited of 137 All 137 (JSON)