# A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization

arXiv: https://arxiv.org/abs/2111.05040 · HTML: https://mlsanity.com/paper/2111.05040/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2021.

## Built on by (most cited first, 30 of 77)

- [Vanilla Bayesian Optimization Performs Great in High Dimensions](https://mlsanity.com/paper/2402.02229/) (2024)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces](https://mlsanity.com/paper/2304.11468/) (2023)
- [Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization](https://mlsanity.com/paper/2210.01628/) (2022)
- [Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spaces](https://mlsanity.com/paper/2307.00618/) (2023)
- [Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial](https://mlsanity.com/paper/2305.04933/) (2023)
- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Macro Placement by Wire-Mask-Guided Black-Box Optimization](https://mlsanity.com/paper/2306.16844/) (2023)
- [A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences](https://mlsanity.com/paper/2406.04739/) (2024)
- [Understanding High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2502.09198/) (2025)
- [Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB](https://mlsanity.com/paper/2303.00890/) (2023)
- [MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences](https://mlsanity.com/paper/2410.02381/) (2024)
- [Offline Multi-Objective Optimization](https://mlsanity.com/paper/2406.03722/) (2024)
- [Distribution Transformers: Fast Approximate Bayesian Inference With On-The-Fly Prior Adaptation](https://mlsanity.com/paper/2502.02463/) (2025)
- [Triangulation candidates for Bayesian optimization](https://mlsanity.com/paper/2112.07457/) (2021)
- [Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes](https://mlsanity.com/paper/2210.07612/) (2022)
- [Trieste: Efficiently Exploring The Depths of Black-box Functions with TensorFlow](https://mlsanity.com/paper/2302.08436/) (2023)
- [Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimization](https://mlsanity.com/paper/2405.15285/) (2024)
- [Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimation](https://mlsanity.com/paper/2307.13371/) (2023)
- [High Dimensional Bayesian Optimization with Kernel Principal Component Analysis](https://mlsanity.com/paper/2204.13753/) (2022)
- [High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategy](https://mlsanity.com/paper/2402.03104/) (2024)
- [Bayesian Safety Validation for Failure Probability Estimation of Black-Box Systems](https://mlsanity.com/paper/2305.02449/) (2023)
- [High-dimensional Bayesian Optimization with Group Testing](https://mlsanity.com/paper/2310.03515/) (2023)
- [Dynamic Online Ensembles of Basis Expansions](https://mlsanity.com/paper/2405.01365/) (2024)
- [BBOPlace-Bench: Benchmarking Black-Box Optimization for Chip Placement](https://mlsanity.com/paper/2510.23472/) (2025)
- [Black-Box Optimization with Implicit Constraints for Public Policy](https://mlsanity.com/paper/2310.18449/) (2023)
- [Implicit Manifold Gaussian Process Regression](https://mlsanity.com/paper/2310.19390/) (2023)
- [Co-orchestration of Multiple Instruments to Uncover Structure-Property Relationships in Combinatorial Libraries](https://mlsanity.com/paper/2402.02198/) (2024)
- [Regional Expected Improvement for Efficient Trust Region Selection in High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2412.11456/) (2024)
- [Safe Bayesian Optimization for the Control of High-Dimensional Embodied Systems](https://mlsanity.com/paper/2412.20350/) (2024)

## Abstract (arXiv)

Bayesian Optimization, the application of Bayesian function approximation to finding optima of expensive functions, has exploded in popularity in recent years. In particular, much attention has been paid to improving its efficiency on problems with many parameters to optimize. This attention has trickled down to the workhorse of high dimensional BO, high dimensional Gaussian process regression, which is also of independent interest. The great flexibility that the Gaussian process prior implies is a boon when modeling complicated, low dimensional surfaces but simply says too little when dimension grows too large. A variety of structural model assumptions have been tested to tame high dimensions, from variable selection and additive decomposition to low dimensional embeddings and beyond. Most of these approaches in turn require modifications of the acquisition function optimization strategy as well. Here we review the defining structural model assumptions and discuss the benefits and drawbacks of these approaches in practice.