mlsanityData to 14 Oct 2026

Transfer Learning for Bayesian Optimization: A Survey

arXiv 2302.05927 ↗First posted 12 Feb 2023cs.LG cs.AIcited by 27

Not on the map. Why?

Abstract (arXiv)

A wide spectrum of design and decision problems, including parameter tuning, A/B testing and drug design, intrinsically are instances of black-box optimization. Bayesian optimization (BO) is a powerful tool that models and optimizes such expensive "black-box" functions. However, at the beginning of optimization, vanilla Bayesian optimization methods often suffer from slow convergence issue due to inaccurate modeling based on few trials. To address this issue, researchers in the BO community propose to incorporate the spirit of transfer learning to accelerate optimization process, which could borrow strength from the past tasks (source tasks) to accelerate the current optimization problem (target task). This survey paper first summarizes transfer learning methods for Bayesian optimization from four perspectives: initial points design, search space design, surrogate model, and acquisition function. Then it highlights its methodological aspects and technical details for each approach. Finally, it showcases a wide range of applications and proposes promising future directions.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 44 of its 131 references that are in the map. First posted on arXiv 12 Feb 2023.

Builds on

44 of 131 references.

References · 44

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  3. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  4. 2015Deep Kernel Learningcited by 445
  5. 2018Conditional Neural Processescited by 426
  6. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  7. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  8. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 364
  9. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
  10. 2018Neural Processescited by 311
  11. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  12. 2019Attentive Neural Processescited by 283
32 more
  1. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 206
  2. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 204
  3. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 195
  4. 2018Auto-Keras: An Efficient Neural Architecture Search Systemcited by 171
  5. 2016Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Roboticscited by 134
  6. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 132
  7. 2014Learning Ordered Representations with Nested Dropoutcited by 84
  8. 2021Few-Shot Bayesian Optimization with Deep Kernel Surrogatescited by 71
  9. 2019Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimizationcited by 70
  10. 2019Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learningcited by 50
  11. 2021OpenBox: A Generalized Black-box Optimization Servicecited by 46
  12. 2018Regret bounds for meta Bayesian optimization with an unknown Gaussian process priorcited by 31
  13. 2020Efficient Automatic CASH via Rising Banditscited by 31
  14. 2018Practical Transfer Learning for Bayesian Optimizationcited by 26
  15. 2021Transfer Learning with Gaussian Processes for Bayesian Optimizationcited by 26
  16. 2021VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decompositioncited by 25
  17. 2021Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimizationcited by 23
  18. 2021Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluationcited by 22
  19. 2016Warm Starting Bayesian Optimizationcited by 21
  20. 2020MUMBO: MUlti-task Max-value Bayesian Optimizationcited by 21
  21. 2019Benchmark and Survey of Automated Machine Learning Frameworkscited by 18
  22. 2020MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurementscited by 18
  23. 2021Hyperparameter Transfer Learning with Adaptive Complexitycited by 17
  24. 2022Towards Dynamic and Safe Configuration Tuning for Cloud Databasescited by 16
  25. 2015Unbounded Bayesian Optimization via Regularizationcited by 13
  26. 2018Hyperparameter Learning via Distributional Transfercited by 13
  27. 2022Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scalecited by 13
  28. 2022Transfer Learning based Search Space Design for Hyperparameter Tuningcited by 13
  29. 2014An Entropy Search Portfolio for Bayesian Optimizationcited by 11
  30. 2022TransBO: Hyperparameter Optimization via Two-Phase Transfer Learningcited by 7
  31. 2017Multiple Adaptive Bayesian Linear Regression for Scalable Bayesian Optimization with Warm Startcited by 4
  32. 2021End-to-End Learning of Deep Kernel Acquisition Functions for Bayesian Optimizationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 8; 2025: 11; 2026: 4.

27 papers.

Sort:

By area:

  1. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 32
  2. 2024Sequential Large Language Model-Based Hyper-parameter Optimizationcited by 24
  3. 2024Evolve Cost-aware Acquisition Functions Using Large Language Modelscited by 17
  4. 2024Reinforced In-Context Black-Box Optimizationcited by 14
  5. 2025Large language models as uncertainty-calibrated optimizers for experimental discoverycited by 13
  6. 2024Machine Learning Insides OptVerse AI Solver: Design Principles and Applicationscited by 7
  7. 2024Monte Carlo Tree Search based Space Transfer for Black-box Optimizationcited by 7
  8. 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddingscited by 7
  9. 2024Efficient Task Transfer for HLS DSEcited by 4
  10. 2024Cost-Sensitive Multi-Fidelity Bayesian Optimization with Transfer of Learning Curve Extrapolationcited by 2
  11. 2025Efficient Visual Appearance Optimization by Learning from Prior Preferencescited by 2
  12. 2025A Hierarchical Surrogate Model for Efficient Multi-Task Parameter Learning in Closed-Loop Controlcited by 2
15 more
  1. 2025Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuningcited by 2
  2. 2023Obeying the Order: Introducing Ordered Transfer Hyperparameter Optimisationcited by 1
  3. 2023Towards General and Efficient Online Tuning for Sparkcited by 1
  4. 2023Data-driven Prior Learning for Bayesian Optimisationcited by 1
  5. 2025Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactionscited by 1
  6. 2025Provable Accelerated Bayesian Optimization with Knowledge Transfercited by 1
  7. 2025Automated Discovery of Laser Dicing Processes with Bayesian Optimization for Semiconductor Manufacturingcited by 1
  8. 2026Default Machine Learning Hyperparameters Do Not Provide Informative Initialization for Bayesian Optimizationcited by 1
  9. 2024Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spacescited by 0
  10. 2025Clustering-based Meta Bayesian Optimization with Theoretical Guaranteecited by 0
  11. 2025Sample-Efficient Bayesian Transfer Learning for Online Machine Parameter Optimizationcited by 0
  12. 2025Offline Multi-Task Multi-Objective Data-Driven Evolutionary Algorithm with Language Surrogate Model and Implicit Q-Learningcited by 0
  13. 2026An Empirical Study on Ensemble-Based Transfer Learning Bayesian Optimisation with Mixed Variable Typescited by 0
  14. 2026Transfer Learning in Bayesian Optimization for Aircraft Designcited by 0
  15. 2026BayMOTH: Bayesian optiMizatiOn with meTa-lookahead -- a simple approacHcited by 0