# Transfer Learning for Bayesian Optimization: A Survey

arXiv: https://arxiv.org/abs/2302.05927 · HTML: https://mlsanity.com/paper/2302.05927/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 131 references that are in the map.
- First posted on arXiv 12 Feb 2023.

## Builds on (most cited first, 30 of 44)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018)
- [Auto-Keras: An Efficient Neural Architecture Search System](https://mlsanity.com/paper/1806.10282/) (2018)
- [Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics](https://mlsanity.com/paper/1602.04450/) (2016)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Learning Ordered Representations with Nested Dropout](https://mlsanity.com/paper/1402.0915/) (2014)
- [Few-Shot Bayesian Optimization with Deep Kernel Surrogates](https://mlsanity.com/paper/2101.07667/) (2021)
- [Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization](https://mlsanity.com/paper/1904.02642/) (2019)
- [Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning](https://mlsanity.com/paper/1909.12552/) (2019)
- [OpenBox: A Generalized Black-box Optimization Service](https://mlsanity.com/paper/2106.00421/) (2021)
- [Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior](https://mlsanity.com/paper/1811.09558/) (2018)
- [Efficient Automatic CASH via Rising Bandits](https://mlsanity.com/paper/2012.04371/) (2020)
- [Practical Transfer Learning for Bayesian Optimization](https://mlsanity.com/paper/1802.02219/) (2018)
- [Transfer Learning with Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2111.11223/) (2021)
- [VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2107.08861/) (2021)
- [Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimization](https://mlsanity.com/paper/2106.04335/) (2021)
- [Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation](https://mlsanity.com/paper/2110.12654/) (2021)

## Built on by (most cited first, 27 of 27)

- [End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes](https://mlsanity.com/paper/2305.15930/) (2023)
- [Sequential Large Language Model-Based Hyper-parameter Optimization](https://mlsanity.com/paper/2410.20302/) (2024)
- [Evolve Cost-aware Acquisition Functions Using Large Language Models](https://mlsanity.com/paper/2404.16906/) (2024)
- [Reinforced In-Context Black-Box Optimization](https://mlsanity.com/paper/2402.17423/) (2024)
- [Large language models as uncertainty-calibrated optimizers for experimental discovery](https://mlsanity.com/paper/2504.06265/) (2025)
- [Machine Learning Insides OptVerse AI Solver: Design Principles and Applications](https://mlsanity.com/paper/2401.05960/) (2024)
- [Monte Carlo Tree Search based Space Transfer for Black-box Optimization](https://mlsanity.com/paper/2412.07186/) (2024)
- [Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings](https://mlsanity.com/paper/2506.07109/) (2025)
- [Efficient Task Transfer for HLS DSE](https://mlsanity.com/paper/2408.13270/) (2024)
- [Cost-Sensitive Multi-Fidelity Bayesian Optimization with Transfer of Learning Curve Extrapolation](https://mlsanity.com/paper/2405.17918/) (2024)
- [Efficient Visual Appearance Optimization by Learning from Prior Preferences](https://mlsanity.com/paper/2507.15355/) (2025)
- [A Hierarchical Surrogate Model for Efficient Multi-Task Parameter Learning in Closed-Loop Control](https://mlsanity.com/paper/2508.12738/) (2025)
- [Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning](https://mlsanity.com/paper/2510.21379/) (2025)
- [Obeying the Order: Introducing Ordered Transfer Hyperparameter Optimisation](https://mlsanity.com/paper/2306.16916/) (2023)
- [Towards General and Efficient Online Tuning for Spark](https://mlsanity.com/paper/2309.01901/) (2023)
- [Data-driven Prior Learning for Bayesian Optimisation](https://mlsanity.com/paper/2311.14653/) (2023)
- [Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactions](https://mlsanity.com/paper/2504.08874/) (2025)
- [Provable Accelerated Bayesian Optimization with Knowledge Transfer](https://mlsanity.com/paper/2511.03125/) (2025)
- [Automated Discovery of Laser Dicing Processes with Bayesian Optimization for Semiconductor Manufacturing](https://mlsanity.com/paper/2511.23141/) (2025)
- [Default Machine Learning Hyperparameters Do Not Provide Informative Initialization for Bayesian Optimization](https://mlsanity.com/paper/2602.08774/) (2026)
- [Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces](https://mlsanity.com/paper/2409.05325/) (2024)
- [Clustering-based Meta Bayesian Optimization with Theoretical Guarantee](https://mlsanity.com/paper/2503.06093/) (2025)
- [Sample-Efficient Bayesian Transfer Learning for Online Machine Parameter Optimization](https://mlsanity.com/paper/2503.15928/) (2025)
- [Offline Multi-Task Multi-Objective Data-Driven Evolutionary Algorithm with Language Surrogate Model and Implicit Q-Learning](https://mlsanity.com/paper/2512.15149/) (2025)
- [An Empirical Study on Ensemble-Based Transfer Learning Bayesian Optimisation with Mixed Variable Types](https://mlsanity.com/paper/2601.15640/) (2026)
- [Transfer Learning in Bayesian Optimization for Aircraft Design](https://mlsanity.com/paper/2603.28999/) (2026)
- [BayMOTH: Bayesian optiMizatiOn with meTa-lookahead -- a simple approacH](https://mlsanity.com/paper/2604.12005/) (2026)

## Abstract (arXiv)

A wide spectrum of design and decision problems, including parameter tuning, A/B testing and drug design, intrinsically are instances of black-box optimization. Bayesian optimization (BO) is a powerful tool that models and optimizes such expensive "black-box" functions. However, at the beginning of optimization, vanilla Bayesian optimization methods often suffer from slow convergence issue due to inaccurate modeling based on few trials. To address this issue, researchers in the BO community propose to incorporate the spirit of transfer learning to accelerate optimization process, which could borrow strength from the past tasks (source tasks) to accelerate the current optimization problem (target task). This survey paper first summarizes transfer learning methods for Bayesian optimization from four perspectives: initial points design, search space design, surrogate model, and acquisition function. Then it highlights its methodological aspects and technical details for each approach. Finally, it showcases a wide range of applications and proposes promising future directions.