# High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups

arXiv: https://arxiv.org/abs/1802.07028 · HTML: https://mlsanity.com/paper/1802.07028/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2018.

## Built on by (most cited first, 30 of 59)

- [Vanilla Bayesian Optimization Performs Great in High Dimensions](https://mlsanity.com/paper/2402.02229/) (2024)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [Gaussian Process Bandit Optimization with Few Batches](https://mlsanity.com/paper/2110.07788/) (2021)
- [Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces](https://mlsanity.com/paper/2102.07188/) (2021)
- [Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization](https://mlsanity.com/paper/2210.01628/) (2022)
- [Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?](https://mlsanity.com/paper/2301.12844/) (2023)
- [Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2402.02746/) (2024)
- [A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences](https://mlsanity.com/paper/2406.04739/) (2024)
- [Parallelizing Thompson Sampling](https://mlsanity.com/paper/2106.01420/) (2021)
- [Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization](https://mlsanity.com/paper/2502.16824/) (2025)
- [Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selection](https://mlsanity.com/paper/2109.09264/) (2021)
- [We Still Don't Understand High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2512.00170/) (2025)
- [Functional Graphical Models: Structure Enables Offline Data-Driven Optimization](https://mlsanity.com/paper/2401.05442/) (2024)
- [Graph Neural Network Bandits](https://mlsanity.com/paper/2207.06456/) (2022)
- ["Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihood](https://mlsanity.com/paper/2405.16730/) (2024)
- [Stochastic Gradient Line Bayesian Optimization for Efficient Noise-Robust Optimization of Parameterized Quantum Circuits](https://mlsanity.com/paper/2111.07952/) (2021)
- [High Dimensional Bayesian Optimization with Kernel Principal Component Analysis](https://mlsanity.com/paper/2204.13753/) (2022)
- [Efficient Planning in Combinatorial Action Spaces with Applications to Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.04376/) (2023)
- [Adversarial Attacks on Gaussian Process Bandits](https://mlsanity.com/paper/2110.08449/) (2021)
- [MBORE: Multi-objective Bayesian Optimisation by Density-Ratio Estimation](https://mlsanity.com/paper/2203.16912/) (2022)
- [Unleashing the Potential of Acquisition Functions in High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2302.08298/) (2023)
- [High Dimensional Bayesian Optimization using Lasso Variable Selection](https://mlsanity.com/paper/2504.01743/) (2025)
- [Active Learning for Level Set Estimation Using Randomized Straddle Algorithms](https://mlsanity.com/paper/2408.03144/) (2024)
- [Cliqueformer: Model-Based Optimization with Structured Transformers](https://mlsanity.com/paper/2410.13106/) (2024)
- [Bayesian Optimization with Clustering and Rollback for CNN Auto Pruning](https://mlsanity.com/paper/2109.10591/) (2021)
- [Test and Evaluation of Quadrupedal Walking Gaits through Sim2Real Gap Quantification](https://mlsanity.com/paper/2201.01323/) (2022)
- [Bayesian Optimization with Informative Covariance](https://mlsanity.com/paper/2208.02704/) (2022)
- [Expected Coordinate Improvement for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2404.11917/) (2024)
- [High-Dimensional Bayesian Optimization via Random Projection of Manifold Subspaces](https://mlsanity.com/paper/2412.16554/) (2024)

## Abstract (arXiv)

Bayesian optimization (BO) is a popular technique for sequential black-box function optimization, with applications including parameter tuning, robotics, environmental monitoring, and more. One of the most important challenges in BO is the development of algorithms that scale to high dimensions, which remains a key open problem despite recent progress. In this paper, we consider the approach of Kandasamy et al. (2015), in which the high-dimensional function decomposes as a sum of lower-dimensional functions on subsets of the underlying variables. In particular, we significantly generalize this approach by lifting the assumption that the subsets are disjoint, and consider additive models with arbitrary overlap among the subsets. By representing the dependencies via a graph, we deduce an efficient message passing algorithm for optimizing the acquisition function. In addition, we provide an algorithm for learning the graph from samples based on Gibbs sampling. We empirically demonstrate the effectiveness of our methods on both synthetic and real-world data.