mlsanityData to 7 Oct 2026

Recent Advances in Bayesian Optimization

arXiv 2206.03301 ↗First posted 7 Jun 2022cs.LG cs.DC cs.NEcited by 74

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization has emerged at the forefront of expensive black-box optimization due to its data efficiency. Recent years have witnessed a proliferation of studies on the development of new Bayesian optimization algorithms and their applications. Hence, this paper attempts to provide a comprehensive and updated survey of recent advances in Bayesian optimization and identify interesting open problems. We categorize the existing work on Bayesian optimization into nine main groups according to the motivations and focus of the proposed algorithms. For each category, we present the main advances with respect to the construction of surrogate models and adaptation of the acquisition functions. Finally, we discuss the open questions and suggest promising future research directions, in particular with regard to heterogeneity, privacy preservation, and fairness in distributed and federated optimization systems.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Builds on 128 of its 321 references that are in the map. First posted on arXiv 7 Jun 2022.

Builds on

128 of 321 references.

References · 128

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  3. 2017A Survey on Multi-Task Learningcited by 838
  4. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  5. 2018A Tutorial on Bayesian Optimizationcited by 734
  6. 2017A Tutorial on Thompson Samplingcited by 420
  7. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  8. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  9. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  10. 2018When Gaussian Process Meets Big Data: A Review of Scalable GPscited by 205
  11. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 202
  12. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 201
38 more
  1. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
  2. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  3. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  4. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
  5. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 131
  6. 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 120
  7. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  8. 2015Batch Bayesian Optimization via Local Penalizationcited by 103
  9. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 103
  10. 2019Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuningcited by 91
  11. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
  12. 2015Predictive Entropy Search for Multi-objective Bayesian Optimizationcited by 88
  13. 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
  14. 2020Mat'ern Gaussian processes on Riemannian manifoldscited by 82
  15. 2020Max-value Entropy Search for Multi-Objective Bayesian Optimization with Constraintscited by 81
  16. 2015Bayesian optimization for materials designcited by 78
  17. 2019Combinatorial Bayesian Optimization using the Graph Cartesian Productcited by 78
  18. 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
  19. 2017Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Spacecited by 76
  20. 2021A survey on high-dimensional Gaussian process modeling with application to Bayesian optimizationcited by 76
  21. 2016The Parallel Knowledge Gradient Method for Batch Bayesian Optimizationcited by 73
  22. 2019Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimizationcited by 70
  23. 2014Input Warping for Bayesian Optimization of Non-stationary Functionscited by 66
  24. 2017Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processescited by 65
  25. 2018Bayesian Optimization of Combinatorial Structurescited by 64
  26. 2019Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variablescited by 64
  27. 2018BOCK : Bayesian Optimization with Cylindrical Kernelscited by 62
  28. 2020Distributionally Robust Bayesian Optimizationcited by 60
  29. 2018High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groupscited by 58
  30. 2019Multi-objective Bayesian Optimization using Pareto-frontier Entropycited by 57
  31. 2019Bayesian Optimisation over Multiple Continuous and Categorical Inputscited by 55
  32. 2021A Simple Approach for Non-stationary Linear Banditscited by 54
  33. 2018A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processescited by 52
  34. 2015Predictive Entropy Search for Bayesian Optimization with Unknown Constraintscited by 51
  35. 2017Batched High-dimensional Bayesian Optimization via Structural Kernel Learningcited by 51
  36. 2013Parallel Gaussian Process Optimization with Upper Confidence Bound and Pure Explorationcited by 50
  37. 2015A General Framework for Constrained Bayesian Optimization using Information-based Searchcited by 50
  38. 2016Time-Varying Gaussian Process Bandit Optimizationcited by 50

50 most cited of 128 All 128 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 15; 2025: 34; 2026: 18.

74 papers.

Sort:

By area:

  1. 2023Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spacescited by 27
  2. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  3. 2023Bayesian Optimization of Catalysis With In-Context Learningcited by 20
  4. 2024Bayesian optimization as a flexible and efficient design framework for sustainable process systemscited by 15
  5. 2025Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMscited by 15
  6. 2024SCOOT: SLO-Oriented Performance Tuning for LLM Inference Enginescited by 9
  7. 2024Goal-Oriented Bayesian Optimal Experimental Design for Nonlinear Models using Markov Chain Monte Carlocited by 8
  8. 2024EARL-BO: Reinforcement Learning for Multi-Step Lookahead, High-Dimensional Bayesian Optimizationcited by 6
  9. 2023Risk-Controlling Model Selection via Guided Bayesian Optimizationcited by 5
  10. 2025HeaRT: A Hierarchical Circuit Reasoning Tree-Based Agentic Framework for AMS Design Optimizationcited by 5
  11. 2022Towards Fairness-Aware Multi-Objective Optimizationcited by 4
  12. 2022A Secure Federated Data-Driven Evolutionary Multi-objective Optimization Algorithmcited by 4
38 more
  1. 2024Model Uncertainty in Evolutionary Optimization and Bayesian Optimization: A Comparative Analysiscited by 4
  2. 2025On the convergence rate of noisy Bayesian Optimization with Expected Improvementcited by 4
  3. 2025Hyperparameter Optimization via Interacting with Probabilistic Circuitscited by 4
  4. 2026LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentationcited by 4
  5. 2024Measurements with Noise: Bayesian Optimization for Co-optimizing Noise and Property Discovery in Automated Experimentscited by 3
  6. 2025Active Alignments of Lens Systems with Reinforcement Learningcited by 3
  7. 2025Deterministic Global Optimization of the Acquisition Function in Bayesian Optimization: To Do or Not To Do?cited by 3
  8. 2025LiBOG: Lifelong Learning for Black-Box Optimizer Generationcited by 3
  9. 2025ChemBOMAS: Accelerated BO in Chemistry with LLM-Enhanced Multi-Agent Systemcited by 3
  10. 2023Gradient and Uncertainty Enhanced Sequential Sampling for Global Fitcited by 2
  11. 2024Expected Coordinate Improvement for High-Dimensional Bayesian Optimizationcited by 2
  12. 2024Optimal Initialization of Batch Bayesian Optimizationcited by 2
  13. 2025BARK: A Fully Bayesian Tree Kernel for Black-box Optimizationcited by 2
  14. 2025Causal-PIK: Causality-based Physical Reasoning with a Physics-Informed Kernelcited by 2
  15. 2025Efficient Visual Appearance Optimization by Learning from Prior Preferencescited by 2
  16. 2025LISTEN to Your Preferences: An LLM Framework for Multi-Objective Selectioncited by 2
  17. 2024Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methodscited by 1
  18. 2024A Tutorial on Gaussian Process Learning-based Model Predictive Controlcited by 1
  19. 2024Bayesian Optimization of Functions over Node Subsets in Graphscited by 1
  20. 2025Causal Discovery via Bayesian Optimizationcited by 1
  21. 2025Stress-Testing ML Pipelines with Adversarial Data Corruptioncited by 1
  22. 2025Black-Box Combinatorial Optimization with Order-Invariant Reinforcement Learningcited by 1
  23. 2025Synergizing chemical and AI communities for advancing laboratories of the futurecited by 1
  24. 2025SemanticOpt: Towards LLM-Based Semantic Black-Box Optimizationcited by 1
  25. 2025A Multidisciplinary Design and Optimization (MDO) Agent Driven by Large Language Modelscited by 1
  26. 2026Advancing Automated Algorithm Design via Evolutionary Stagewise Design with LLMscited by 1
  27. 2026Human-in-the-Loop Meta Bayesian Optimization for Fusion Energy and Scientific Applicationscited by 1
  28. 2026Unified Audio Intelligence Without Regressing on Text Intelligencecited by 1
  29. 2023Exact and general decoupled solutions of the LMC Multitask Gaussian Process modelcited by 0
  30. 2024Simulation Based Bayesian Optimizationcited by 0
  31. 2024On Constructing Algorithm Portfolios in Algorithm Selection for Computationally Expensive Black-box Optimization in the Fixed-budget Settingcited by 0
  32. 2024Scalable Batch Bayesian Optimization Via Subspace Acquisition Functionscited by 0
  33. 2025Sample-Efficient Bayesian Transfer Learning for Online Machine Parameter Optimizationcited by 0
  34. 2025Why risk matters for protein binder designcited by 0
  35. 2025Multi-Variable Batch Bayesian Optimization in Materials Research: Synthetic Data Analysis of Noise Sensitivity and Problem Landscape Effectscited by 0
  36. 2025A Balanced Approach of Rapid Genetic Exploration and Surrogate Exploitation for Hyperparameter Optimizationcited by 0
  37. 2025Gradient-based Sample Selection for Faster Bayesian Optimizationcited by 0
  38. 2025crowd-hpo: Realistic Hyperparameter Optimization and Benchmarking for Learning from Crowds with Noisy Labelscited by 0

50 most cited of 74 All 74 (JSON)