mlsanityData to 7 Oct 2026

Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes

arXiv 2203.01459 ↗First posted 2 Mar 2022cs.LG stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization is a technique for optimizing black-box target functions. At the core of Bayesian optimization is a surrogate model that predicts the output of the target function at previously unseen inputs to facilitate the selection of promising input values. Gaussian processes (GPs) are commonly used as surrogate models but are known to scale poorly with the number of observations. We adapt the Vecchia approximation, a popular GP approximation from spatial statistics, to enable scalable high-dimensional Bayesian optimization. We develop several improvements and extensions, including training warped GPs using mini-batch gradient descent, approximate neighbor search, and selecting multiple input values in parallel. We focus on the use of our warped Vecchia GP in trust-region Bayesian optimization via Thompson sampling. On several test functions and on two reinforcement-learning problems, our methods compared favorably to the state of the art.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 28 of its 54 references that are in the map. First posted on arXiv 2 Mar 2022.

Builds on

28 of 54 references.

References · 28

  1. 2017Billion-scale similarity search with GPUscited by 2,477
  2. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  3. 2018A Tutorial on Bayesian Optimizationcited by 734
  4. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  5. 2015Deep Kernel Learningcited by 440
  6. 2013Gaussian Processes for Big Datacited by 377
  7. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  8. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
  9. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  10. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
  11. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  12. 2018Maximizing acquisition functions for Bayesian optimizationcited by 113
16 more
  1. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 103
  2. 2016The Parallel Knowledge Gradient Method for Batch Bayesian Optimizationcited by 73
  3. 2014Input Warping for Bayesian Optimization of Non-stationary Functionscited by 66
  4. 2014Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictionscited by 66
  5. 2017A general framework for Vecchia approximations of Gaussian processescited by 66
  6. 2017The reparameterization trick for acquisition functionscited by 61
  7. 2016Parallel Bayesian Global Optimization of Expensive Functionscited by 43
  8. 2020Sparse Cholesky factorization by Kullback-Leibler minimizationcited by 40
  9. 2016Permutation and Grouping Methods for Sharpening Gaussian Process Approximationscited by 24
  10. 2021Conditioning Sparse Variational Gaussian Processes for Online Decision-makingcited by 23
  11. 2018Vecchia approximations of Gaussian-process predictionscited by 21
  12. 2020Sparse within Sparse Gaussian Processes using Neighbor Informationcited by 18
  13. 2020Scaled Vecchia approximation for fast computer-model emulationcited by 15
  14. 2021Correlation-based sparse inverse Cholesky factorization for fast Gaussian-process inferencecited by 12
  15. 2022Variational Nearest Neighbor Gaussian Processcited by 8
  16. 2021Correlated Product of Experts for Sparse Gaussian Process Regressioncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 1; 2025: 4; 2026: 3.

11 papers.

  1. 2024Scalable Bayesian Optimization via Focalized Sparse Gaussian Processescited by 8
  2. 2025Collaborative Bayesian Optimization via Wasserstein Barycenterscited by 7
  3. 2023Vecchia Gaussian Process Ensembles on Internal Representations of Deep Neural Networkscited by 3
  4. 2023Scalable Model-Based Gaussian Process Clusteringcited by 1
  5. 2022Optimistic Optimization of Gaussian Process Samplescited by 0
  6. 2025Gradient-based Sample Selection for Faster Bayesian Optimizationcited by 0
  7. 2025Taking the GP Out of the Loopcited by 0
  8. 2025BOOST: A Data-Driven Framework for the Automated Joint Selection of Kernel and Acquisition Functions in Bayesian Optimizationcited by 0
  9. 2026Amortized Acquisition Optimization for Bayesian Optimization with Variational Mutual Informationcited by 0
  10. 2026Bayesian Optimization in Linear Timecited by 0
  11. 2026Scalable Derivative Gaussian Processes via Exact Gradient Reductioncited by 0