# Transfer Learning with Gaussian Processes for Bayesian Optimization

arXiv: https://arxiv.org/abs/2111.11223 · HTML: https://mlsanity.com/paper/2111.11223/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 22 Nov 2021.

## Builds on (most cited first, 9 of 9)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Multi-Information Source Optimization](https://mlsanity.com/paper/1603.00389/) (2016)
- [A Literature Survey of Benchmark Functions For Global Optimization Problems](https://mlsanity.com/paper/1308.4008/) (2013)
- [Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization](https://mlsanity.com/paper/1904.02642/) (2019)
- [Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variables](https://mlsanity.com/paper/1910.01688/) (2019)
- [Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization](https://mlsanity.com/paper/1703.01250/) (2017)
- [Emulation of physical processes with Emukit](https://mlsanity.com/paper/2110.13293/) (2021)
- [Transferring Knowledge across Learning Processes](https://mlsanity.com/paper/1812.01054/) (2018)
- [Automatic Exploration of Machine Learning Experiments on OpenML](https://mlsanity.com/paper/1806.10961/) (2018)

## Built on by (most cited first, 26 of 26)

- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Transfer Learning for Bayesian Optimization: A Survey](https://mlsanity.com/paper/2302.05927/) (2023)
- [Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction](https://mlsanity.com/paper/2205.02708/) (2022)
- [Efficient Black-box Adversarial Attacks via Bayesian Optimization Guided by a Function Prior](https://mlsanity.com/paper/2405.19098/) (2024)
- [Monte Carlo Tree Search based Space Transfer for Black-box Optimization](https://mlsanity.com/paper/2412.07186/) (2024)
- [A Scalable Test Problem Generator for Sequential Transfer Optimization](https://mlsanity.com/paper/2304.08503/) (2023)
- [Scalable Meta-Learning with Gaussian Processes](https://mlsanity.com/paper/2312.00742/) (2023)
- [Active Alignments of Lens Systems with Reinforcement Learning](https://mlsanity.com/paper/2503.02075/) (2025)
- [None To Optima in Few Shots: Bayesian Optimization with MDP Priors](https://mlsanity.com/paper/2511.01006/) (2025)
- [MALIBO: Meta-learning for Likelihood-free Bayesian Optimization](https://mlsanity.com/paper/2307.03565/) (2023)
- [Multi-Objective Multi-Fidelity Bayesian Optimization with Causal Priors](https://mlsanity.com/paper/2602.00788/) (2026)
- [Combining additivity and active subspaces for high-dimensional Gaussian process modeling](https://mlsanity.com/paper/2402.03809/) (2024)
- [Global Safe Sequential Learning via Efficient Knowledge Transfer](https://mlsanity.com/paper/2402.14402/) (2024)
- [Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactions](https://mlsanity.com/paper/2504.08874/) (2025)
- [Provable Accelerated Bayesian Optimization with Knowledge Transfer](https://mlsanity.com/paper/2511.03125/) (2025)
- [Default Machine Learning Hyperparameters Do Not Provide Informative Initialization for Bayesian Optimization](https://mlsanity.com/paper/2602.08774/) (2026)
- [Data-Efficient Strategies for Probabilistic Voltage Envelopes under Network Contingencies](https://mlsanity.com/paper/2310.00763/) (2023)
- [Transfer Learning Bayesian Optimization to Design Competitor DNA Molecules for Use in Diagnostic Assays](https://mlsanity.com/paper/2402.17704/) (2024)
- [Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces](https://mlsanity.com/paper/2409.05325/) (2024)
- [Clustering-based Meta Bayesian Optimization with Theoretical Guarantee](https://mlsanity.com/paper/2503.06093/) (2025)
- [Sample-Efficient Bayesian Transfer Learning for Online Machine Parameter Optimization](https://mlsanity.com/paper/2503.15928/) (2025)
- [Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization](https://mlsanity.com/paper/2505.11294/) (2025)
- [An Empirical Study on Ensemble-Based Transfer Learning Bayesian Optimisation with Mixed Variable Types](https://mlsanity.com/paper/2601.15640/) (2026)
- [SMOG: Scalable Meta-Learning for Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2601.22131/) (2026)
- [Weighted Wasserstein Barycenter of Gaussian Processes for exotic Bayesian Optimization tasks](https://mlsanity.com/paper/2602.09181/) (2026)
- [MF-SCBO : Multi-fidelity Scalable Constrained Bayesian Optimization](https://mlsanity.com/paper/2609.29941/) (2026)

## Abstract (arXiv)

Bayesian optimization is a powerful paradigm to optimize black-box functions based on scarce and noisy data. Its data efficiency can be further improved by transfer learning from related tasks. While recent transfer models meta-learn a prior based on large amount of data, in the low-data regime methods that exploit the closed-form posterior of Gaussian processes (GPs) have an advantage. In this setting, several analytically tractable transfer-model posteriors have been proposed, but the relative advantages of these methods are not well understood. In this paper, we provide a unified view on hierarchical GP models for transfer learning, which allows us to analyze the relationship between methods. As part of the analysis, we develop a novel closed-form boosted GP transfer model that fits between existing approaches in terms of complexity. We evaluate the performance of the different approaches in large-scale experiments and highlight strengths and weaknesses of the different transfer-learning methods.