mlsanityData to 7 Oct 2026

Approximation-Aware Bayesian Optimization

arXiv 2406.04308 ↗First posted 6 Jun 2024cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

High-dimensional Bayesian optimization (BO) tasks such as molecular design often require 10,000 function evaluations before obtaining meaningful results. While methods like sparse variational Gaussian processes (SVGPs) reduce computational requirements in these settings, the underlying approximations result in suboptimal data acquisitions that slow the progress of optimization. In this paper we modify SVGPs to better align with the goals of BO: targeting informed data acquisition rather than global posterior fidelity. Using the framework of utility-calibrated variational inference, we unify GP approximation and data acquisition into a joint optimization problem, thereby ensuring optimal decisions under a limited computational budget. Our approach can be used with any decision-theoretic acquisition function and is compatible with trust region methods like TuRBO. We derive efficient joint objectives for the expected improvement and knowledge gradient acquisition functions in both the standard and batch BO settings. Our approach outperforms standard SVGPs on high-dimensional benchmark tasks in control and molecular design.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 34 of its 67 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

34 of 67 references.

References · 34

  1. 2016Variational Inference: A Review for Statisticianscited by 1,466
  2. 2018A Tutorial on Bayesian Optimizationcited by 734
  3. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  4. 2013Gaussian Processes for Big Datacited by 377
  5. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  6. 2013Black Box Variational Inferencecited by 317
  7. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 294
  8. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  9. 2013Expectation Propagation for approximate Bayesian inferencecited by 172
  10. 2013A General Framework for Updating Belief Distributionscited by 172
  11. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 121
  12. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
22 more
  1. 2018Maximizing acquisition functions for Bayesian optimizationcited by 113
  2. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 99
  3. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
  4. 2017Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Spacecited by 76
  5. 2017Frequentist Consistency of Variational Bayescited by 60
  6. 2017Bayesian Optimization with Gradientscited by 59
  7. 2015On Sparse variational methods and the Kullback-Leibler divergence between stochastic processescited by 56
  8. 2021LassoBench: A High-Dimensional Hyperparameter Optimization Benchmark Suite for Lassocited by 41
  9. 2020Causal Bayesian Optimizationcited by 39
  10. 2020Scalable Thompson Sampling using Sparse Gaussian Process Modelscited by 39
  11. 2023Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisationcited by 32
  12. 2022Discovering Many Diverse Solutions with Bayesian Optimizationcited by 29
  13. 2021Conditioning Sparse Variational Gaussian Processes for Online Decision-makingcited by 23
  14. 2018Loss-Calibrated Approximate Inference in Bayesian Neural Networkscited by 18
  15. 2019Variational Bayesian Decision-making for Continuous Utilitiescited by 15
  16. 2020Direct loss minimization algorithms for sparse Gaussian processescited by 10
  17. 2021Adaptive Inducing Points Selection For Gaussian Processescited by 8
  18. 2022Loss-calibrated expectation propagation for approximate Bayesian decision-makingcited by 5
  19. 2022Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Treescited by 5
  20. 2019Asymptotic Consistency of Loss-Calibrated Variational Bayescited by 3
  21. 2019On the Statistical Consistency of Risk-Sensitive Bayesian Decision-Makingcited by 2
  22. 2023Long-tailed Classification from a Bayesian-decision-theory Perspectivecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 3.

8 papers.

  1. 2025Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spacescited by 13
  2. 2025We Still Don't Understand High-Dimensional Bayesian Optimizationcited by 9
  3. 2024Scalable Bayesian Optimization via Focalized Sparse Gaussian Processescited by 8
  4. 2025Reasoning BO: Enhancing Bayesian Optimization with Long-Context Reasoning Power of LLMscited by 8
  5. 2026BayeSQP: Bayesian Optimization through Sequential Quadratic Programmingcited by 5
  6. 2026Efficient Adaptive Data Acquisition via Pretrained Belief Representationscited by 2
  7. 2025Robust and Computation-Aware Gaussian Processescited by 1
  8. 2026Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputscited by 1