mlsanityData to 7 Oct 2026

The Parallel Knowledge Gradient Method for Batch Bayesian Optimization

arXiv 1606.04414 ↗First posted 14 Jun 2016stat.ML cs.AI cs.LGcited by 73

Not on the map. Why?

Abstract (arXiv)

In many applications of black-box optimization, one can evaluate multiple points simultaneously, e.g. when evaluating the performances of several different neural network architectures in a parallel computing environment. In this paper, we develop a novel batch Bayesian optimization algorithm --- the parallel knowledge gradient method. By construction, this method provides the one-step Bayes-optimal batch of points to sample. We provide an efficient strategy for computing this Bayes-optimal batch of points, and we demonstrate that the parallel knowledge gradient method finds global optima significantly faster than previous batch Bayesian optimization algorithms on both synthetic test functions and when tuning hyperparameters of practical machine learning algorithms, especially when function evaluations are noisy.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 17; 2023: 21; 2024: 7; 2025: 6; 2026: 9.

73 papers.

Sort:

By area:

  1. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  2. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 144
  3. 2022Recent Advances in Bayesian Optimizationcited by 74
  4. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 70
  5. 2021High-Dimensional Experimental Design and Kernel Banditscited by 55
  6. 2022Pareto Set Learning for Expensive Multi-Objective Optimizationcited by 51
  7. 2021GIBBON: General-purpose Information-Based Bayesian OptimisatioNcited by 47
  8. 2021Bayesian Optimization of Function Networkscited by 33
  9. 2022Thinking inside the box: A tutorial on grey-box Bayesian optimizationcited by 26
  10. 2021Conditioning Sparse Variational Gaussian Processes for Online Decision-makingcited by 23
  11. 2023qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimizationcited by 23
  12. 2023Framework and Benchmarks for Combinatorial and Mixed-variable Bayesian Optimizationcited by 15
38 more
  1. 2022Fair and Green Hyperparameter Optimization via Multi-objective and Multiple Information Source Bayesian Optimizationcited by 13
  2. 2021Faster Improvement Rate Population Based Trainingcited by 12
  3. 2022Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processescited by 11
  4. 2022Generalizing Bayesian Optimization with Decision-theoretic Entropiescited by 11
  5. 2023Memory-Based Dual Gaussian Processes for Sequential Learningcited by 11
  6. 2023qPOTS: Efficient batch multiobjective Bayesian optimization via Pareto optimal Thompson samplingcited by 11
  7. 2023Bayesian Optimization of Function Networks with Partial Evaluationscited by 10
  8. 2021MetaTune: Meta-Learning Based Cost Model for Fast and Efficient Auto-tuning Frameworkscited by 9
  9. 2021A portfolio approach to massively parallel Bayesian optimizationcited by 9
  10. 2023Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Designcited by 9
  11. 2022PropertyDAG: Multi-objective Bayesian optimization of partially ordered, mixed-variable properties for biological sequence designcited by 8
  12. 2022Fast online inference for nonlinear contextual bandit based on Generative Adversarial Networkcited by 7
  13. 2022Fantasizing with Dual GPs in Bayesian Optimization and Active Learningcited by 7
  14. 2023Gaussian Max-Value Entropy Search for Multi-Agent Bayesian Optimizationcited by 7
  15. 2023Batch Bayesian Optimization for Replicable Experimental Designcited by 7
  16. 2024Function-space Parameterization of Neural Networks for Sequential Learningcited by 7
  17. 2025Collaborative Bayesian Optimization via Wasserstein Barycenterscited by 7
  18. 2021Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networkscited by 6
  19. 2026Adaptive Acquisition Selection for Bayesian Optimization with Large Language Modelscited by 6
  20. 2021Batch Bayesian Optimization on Permutations using the Acquisition Weighted Kernelcited by 5
  21. 2022Adjusted Expected Improvement for Cumulative Regret Minimization in Noisy Bayesian Optimizationcited by 5
  22. 2023Knowledge Gradient for Multi-Objective Bayesian Optimization with Decoupled Evaluationscited by 5
  23. 2023Unleashing the Potential of Acquisition Functions in High-Dimensional Bayesian Optimizationcited by 5
  24. 2023CO-BED: Information-Theoretic Contextual Optimization via Bayesian Experimental Designcited by 4
  25. 2023Pseudo-Bayesian Optimizationcited by 4
  26. 2024Uncertainty-Guided Likelihood Tree Searchcited by 4
  27. 2022Bayesian Calibration for Activity Based Modelscited by 3
  28. 2022On the Finite-Time Performance of the Knowledge Gradient Algorithmcited by 3
  29. 2023Online simulator-based experimental design for cognitive model selectioncited by 3
  30. 2021A New Knowledge Gradient-based Method for Constrained Bayesian Optimizationcited by 2
  31. 2022Targeted Adaptive Designcited by 2
  32. 2022Batch Bayesian Optimization via Particle Gradient Flowscited by 2
  33. 2022Multi-step Planning for Automated Hyperparameter Optimization with OptFormercited by 2
  34. 2023Scalable Bayesian optimization with high-dimensional outputs using randomized prior networkscited by 2
  35. 2023Is novelty predictable?cited by 2
  36. 2023On Active Learning for Gaussian Process-based Global Sensitivity Analysiscited by 2
  37. 2023Hyperparameter Adaptive Search for Surrogate Optimization: A Self-Adjusting Approachcited by 2
  38. 2024Expected Diverse Utility (EDU): Diverse Bayesian Optimization of Expensive Computer Simulatorscited by 2

50 most cited of 73 All 73 (JSON)