# Recent Advances in Bayesian Optimization

arXiv: https://arxiv.org/abs/2206.03301 · HTML: https://mlsanity.com/paper/2206.03301/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Builds on 128 of its 321 references that are in the map.
- First posted on arXiv 7 Jun 2022.

## Builds on (most cited first, 30 of 128)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018)
- [Bayesian Optimization in a Billion Dimensions via Random Embeddings](https://mlsanity.com/paper/1301.1942/) (2013)
- [High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces](https://mlsanity.com/paper/2103.00349/) (2021)
- [High Dimensional Bayesian Optimisation and Bandits via Additive Models](https://mlsanity.com/paper/1503.01673/) (2015)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Multi-fidelity Bayesian Optimisation with Continuous Approximations](https://mlsanity.com/paper/1703.06240/) (2017)
- [Batched Large-scale Bayesian Optimization in High-dimensional Spaces](https://mlsanity.com/paper/1706.01445/) (2017)
- [Batch Bayesian Optimization via Local Penalization](https://mlsanity.com/paper/1505.08052/) (2015)
- [Constrained Bayesian Optimization with Noisy Experiments](https://mlsanity.com/paper/1706.07094/) (2017)
- [Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning](https://mlsanity.com/paper/1903.04703/) (2019)
- [Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2001.11659/) (2020)
- [Predictive Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/1511.05467/) (2015)
- [Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces](https://mlsanity.com/paper/2109.10964/) (2021)
- [Mat'ern Gaussian processes on Riemannian manifolds](https://mlsanity.com/paper/2006.10160/) (2020)
- [Max-value Entropy Search for Multi-Objective Bayesian Optimization with Constraints](https://mlsanity.com/paper/2009.01721/) (2020)
- [Bayesian optimization for materials design](https://mlsanity.com/paper/1506.01349/) (2015)
- [Combinatorial Bayesian Optimization using the Graph Cartesian Product](https://mlsanity.com/paper/1902.00448/) (2019)
- [Federated Bayesian Optimization via Thompson Sampling](https://mlsanity.com/paper/2010.10154/) (2020)

## Built on by (most cited first, 30 of 76)

- [Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spaces](https://mlsanity.com/paper/2307.00618/) (2023)
- [A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences](https://mlsanity.com/paper/2406.04739/) (2024)
- [Bayesian Optimization of Catalysis With In-Context Learning](https://mlsanity.com/paper/2304.05341/) (2023)
- [Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs](https://mlsanity.com/paper/2509.17998/) (2025)
- [Bayesian optimization as a flexible and efficient design framework for sustainable process systems](https://mlsanity.com/paper/2401.16373/) (2024)
- [Goal-Oriented Bayesian Optimal Experimental Design for Nonlinear Models using Markov Chain Monte Carlo](https://mlsanity.com/paper/2403.18072/) (2024)
- [SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines](https://mlsanity.com/paper/2408.04323/) (2024)
- [EARL-BO: Reinforcement Learning for Multi-Step Lookahead, High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2411.00171/) (2024)
- [LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation](https://mlsanity.com/paper/2605.22054/) (2026)
- [Risk-Controlling Model Selection via Guided Bayesian Optimization](https://mlsanity.com/paper/2312.01692/) (2023)
- [HeaRT: A Hierarchical Circuit Reasoning Tree-Based Agentic Framework for AMS Design Optimization](https://mlsanity.com/paper/2511.19669/) (2025)
- [Towards Fairness-Aware Multi-Objective Optimization](https://mlsanity.com/paper/2207.12138/) (2022)
- [A Secure Federated Data-Driven Evolutionary Multi-objective Optimization Algorithm](https://mlsanity.com/paper/2210.08295/) (2022)
- [Model Uncertainty in Evolutionary Optimization and Bayesian Optimization: A Comparative Analysis](https://mlsanity.com/paper/2403.14413/) (2024)
- [On the convergence rate of noisy Bayesian Optimization with Expected Improvement](https://mlsanity.com/paper/2501.09262/) (2025)
- [Hyperparameter Optimization via Interacting with Probabilistic Circuits](https://mlsanity.com/paper/2505.17804/) (2025)
- [Measurements with Noise: Bayesian Optimization for Co-optimizing Noise and Property Discovery in Automated Experiments](https://mlsanity.com/paper/2410.02717/) (2024)
- [Active Alignments of Lens Systems with Reinforcement Learning](https://mlsanity.com/paper/2503.02075/) (2025)
- [Deterministic Global Optimization of the Acquisition Function in Bayesian Optimization: To Do or Not To Do?](https://mlsanity.com/paper/2503.03625/) (2025)
- [BARK: A Fully Bayesian Tree Kernel for Black-box Optimization](https://mlsanity.com/paper/2503.05574/) (2025)
- [LiBOG: Lifelong Learning for Black-Box Optimizer Generation](https://mlsanity.com/paper/2505.13025/) (2025)
- [ChemBOMAS: Accelerated BO in Chemistry with LLM-Enhanced Multi-Agent System](https://mlsanity.com/paper/2509.08736/) (2025)
- [Efficient and Principled Scientific Discovery through Bayesian Optimization: A Tutorial](https://mlsanity.com/paper/2604.01328/) (2026)
- [Gradient and Uncertainty Enhanced Sequential Sampling for Global Fit](https://mlsanity.com/paper/2310.00110/) (2023)
- [Expected Coordinate Improvement for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2404.11917/) (2024)
- [Optimal Initialization of Batch Bayesian Optimization](https://mlsanity.com/paper/2404.17997/) (2024)
- [Causal-PIK: Causality-based Physical Reasoning with a Physics-Informed Kernel](https://mlsanity.com/paper/2505.22861/) (2025)
- [Efficient Visual Appearance Optimization by Learning from Prior Preferences](https://mlsanity.com/paper/2507.15355/) (2025)
- [LISTEN to Your Preferences: An LLM Framework for Multi-Objective Selection](https://mlsanity.com/paper/2510.25799/) (2025)
- [Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methods](https://mlsanity.com/paper/2403.08352/) (2024)

## Abstract (arXiv)

Bayesian optimization has emerged at the forefront of expensive black-box optimization due to its data efficiency. Recent years have witnessed a proliferation of studies on the development of new Bayesian optimization algorithms and their applications. Hence, this paper attempts to provide a comprehensive and updated survey of recent advances in Bayesian optimization and identify interesting open problems. We categorize the existing work on Bayesian optimization into nine main groups according to the motivations and focus of the proposed algorithms. For each category, we present the main advances with respect to the construction of surrogate models and adaptation of the acquisition functions. Finally, we discuss the open questions and suggest promising future research directions, in particular with regard to heterogeneity, privacy preservation, and fairness in distributed and federated optimization systems.