# Practical Transfer Learning for Bayesian Optimization

arXiv: https://arxiv.org/abs/1802.02219 · HTML: https://mlsanity.com/paper/1802.02219/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2018.

## Built on by (most cited first, 26 of 26)

- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://mlsanity.com/paper/2204.11051/) (2022)
- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?](https://mlsanity.com/paper/2503.22674/) (2025)
- [YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization](https://mlsanity.com/paper/2109.03670/) (2021)
- [PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning](https://mlsanity.com/paper/2306.12370/) (2023)
- [Transfer Learning for Bayesian Optimization: A Survey](https://mlsanity.com/paper/2302.05927/) (2023)
- [A General Framework for User-Guided Bayesian Optimization](https://mlsanity.com/paper/2311.14645/) (2023)
- [Triangulation candidates for Bayesian optimization](https://mlsanity.com/paper/2112.07457/) (2021)
- [Speeding Up Multi-Objective Hyperparameter Optimization by Task Similarity-Based Meta-Learning for the Tree-Structured Parzen Estimator](https://mlsanity.com/paper/2212.06751/) (2022)
- [HPC Storage Service Autotuning Using Variational-Autoencoder-Guided Asynchronous Bayesian Optimization](https://mlsanity.com/paper/2210.00798/) (2022)
- [None To Optima in Few Shots: Bayesian Optimization with MDP Priors](https://mlsanity.com/paper/2511.01006/) (2025)
- [MALIBO: Meta-learning for Likelihood-free Bayesian Optimization](https://mlsanity.com/paper/2307.03565/) (2023)
- [Efficient Bayesian Optimization with Deep Kernel Learning and Transformer Pre-trained on Multiple Heterogeneous Datasets](https://mlsanity.com/paper/2308.04660/) (2023)
- [Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization](https://mlsanity.com/paper/2511.02570/) (2025)
- [Federated Covariate Shift Adaptation for Missing Target Output Values](https://mlsanity.com/paper/2302.14427/) (2023)
- [Obeying the Order: Introducing Ordered Transfer Hyperparameter Optimisation](https://mlsanity.com/paper/2306.16916/) (2023)
- [Derivation of Output Correlation Inferences for Multi-Output (aka Multi-Task) Gaussian Process](https://mlsanity.com/paper/2501.07964/) (2025)
- [Provable Accelerated Bayesian Optimization with Knowledge Transfer](https://mlsanity.com/paper/2511.03125/) (2025)
- [Karasu: A Collaborative Approach to Efficient Cluster Configuration for Big Data Analytics](https://mlsanity.com/paper/2308.11792/) (2023)
- [SMOG: Scalable Meta-Learning for Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2601.22131/) (2026)
- [Weighted Wasserstein Barycenter of Gaussian Processes for exotic Bayesian Optimization tasks](https://mlsanity.com/paper/2602.09181/) (2026)
- [Empirical Gaussian Processes](https://mlsanity.com/paper/2602.12082/) (2026)
- [Transfer Learning in Bayesian Optimization for Aircraft Design](https://mlsanity.com/paper/2603.28999/) (2026)
- [Regime-Conditioned Evaluation in Multi-Context Bayesian Optimization](https://mlsanity.com/paper/2605.04895/) (2026)
- [Oracle Supervision Transfers for Hyperparameter Prediction in Model-Based Image Denoising](https://mlsanity.com/paper/2605.20479/) (2026)

## Abstract (arXiv)

When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemble model for Bayesian optimization that is a generalization of two existing transfer learning extensions to Bayesian optimization and establish a worst-case bound compared to vanilla Bayesian optimization. Using a large collection of hyperparameter optimization benchmark problems, we demonstrate that our contributions substantially reduce optimization time compared to standard Gaussian process-based Bayesian optimization and improve over the current state-of-the-art for transfer hyperparameter optimization.