# The Parallel Knowledge Gradient Method for Batch Bayesian Optimization

arXiv: https://arxiv.org/abs/1606.04414 · HTML: https://mlsanity.com/paper/1606.04414/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2016.

## Built on by (most cited first, 30 of 74)

- [Unexpected Improvements to Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2310.20708/) (2023)
- [Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020](https://mlsanity.com/paper/2104.10201/) (2021)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [High-Dimensional Experimental Design and Kernel Bandits](https://mlsanity.com/paper/2105.05806/) (2021)
- [Pareto Set Learning for Expensive Multi-Objective Optimization](https://mlsanity.com/paper/2210.08495/) (2022)
- [GIBBON: General-purpose Information-Based Bayesian OptimisatioN](https://mlsanity.com/paper/2102.03324/) (2021)
- [Bayesian Optimization of Function Networks](https://mlsanity.com/paper/2112.15311/) (2021)
- [Thinking inside the box: A tutorial on grey-box Bayesian optimization](https://mlsanity.com/paper/2201.00272/) (2022)
- [Conditioning Sparse Variational Gaussian Processes for Online Decision-making](https://mlsanity.com/paper/2110.15172/) (2021)
- [qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization](https://mlsanity.com/paper/2303.15746/) (2023)
- [Framework and Benchmarks for Combinatorial and Mixed-variable Bayesian Optimization](https://mlsanity.com/paper/2306.09803/) (2023)
- [Fair and Green Hyperparameter Optimization via Multi-objective and Multiple Information Source Bayesian Optimization](https://mlsanity.com/paper/2205.08835/) (2022)
- [Faster Improvement Rate Population Based Training](https://mlsanity.com/paper/2109.13800/) (2021)
- [Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes](https://mlsanity.com/paper/2203.01459/) (2022)
- [Generalizing Bayesian Optimization with Decision-theoretic Entropies](https://mlsanity.com/paper/2210.01383/) (2022)
- [Memory-Based Dual Gaussian Processes for Sequential Learning](https://mlsanity.com/paper/2306.03566/) (2023)
- [qPOTS: Efficient batch multiobjective Bayesian optimization via Pareto optimal Thompson sampling](https://mlsanity.com/paper/2310.15788/) (2023)
- [Bayesian Optimization of Function Networks with Partial Evaluations](https://mlsanity.com/paper/2311.02146/) (2023)
- [MetaTune: Meta-Learning Based Cost Model for Fast and Efficient Auto-tuning Frameworks](https://mlsanity.com/paper/2102.04199/) (2021)
- [A portfolio approach to massively parallel Bayesian optimization](https://mlsanity.com/paper/2110.09334/) (2021)
- [Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design](https://mlsanity.com/paper/2306.14348/) (2023)
- [Adaptive Acquisition Selection for Bayesian Optimization with Large Language Models](https://mlsanity.com/paper/2602.07904/) (2026)
- [PropertyDAG: Multi-objective Bayesian optimization of partially ordered, mixed-variable properties for biological sequence design](https://mlsanity.com/paper/2210.04096/) (2022)
- [Batch Bayesian Optimization for Replicable Experimental Design](https://mlsanity.com/paper/2311.01195/) (2023)
- [Function-space Parameterization of Neural Networks for Sequential Learning](https://mlsanity.com/paper/2403.10929/) (2024)
- [Fast online inference for nonlinear contextual bandit based on Generative Adversarial Network](https://mlsanity.com/paper/2202.08867/) (2022)
- [Fantasizing with Dual GPs in Bayesian Optimization and Active Learning](https://mlsanity.com/paper/2211.01053/) (2022)
- [Gaussian Max-Value Entropy Search for Multi-Agent Bayesian Optimization](https://mlsanity.com/paper/2303.05694/) (2023)
- [Collaborative Bayesian Optimization via Wasserstein Barycenters](https://mlsanity.com/paper/2504.10770/) (2025)

## Abstract (arXiv)

In many applications of black-box optimization, one can evaluate multiple points simultaneously, e.g. when evaluating the performances of several different neural network architectures in a parallel computing environment. In this paper, we develop a novel batch Bayesian optimization algorithm --- the parallel knowledge gradient method. By construction, this method provides the one-step Bayes-optimal batch of points to sample. We provide an efficient strategy for computing this Bayes-optimal batch of points, and we demonstrate that the parallel knowledge gradient method finds global optima significantly faster than previous batch Bayesian optimization algorithms on both synthetic test functions and when tuning hyperparameters of practical machine learning algorithms, especially when function evaluations are noisy.