mlsanityData to 14 Oct 2026

Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces

arXiv 2409.05325 ↗First posted 9 Sep 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box functions. However, in settings with very few function evaluations, a successful application of BO may require transferring information from historical experiments. These related experiments may not have exactly the same tunable parameters (search spaces), motivating the need for BO with transfer learning for heterogeneous search spaces. In this paper, we propose two methods for this setting. The first approach leverages a Gaussian process (GP) model with a conditional kernel to transfer information between different search spaces. Our second approach treats the missing parameters as hyperparameters of the GP model that can be inferred jointly with the other GP hyperparameters or set to fixed values. We show that these two methods perform well on several benchmark problems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 40 references that are in the map. First posted on arXiv 9 Sep 2024.

Builds on

17 of 40 references.

References · 17

  1. 2018A Tutorial on Bayesian Optimizationcited by 744
  2. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 149
  3. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 148
  4. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 104
  5. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 92
  6. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  7. 2017Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Spacecited by 76
  8. 2021HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenMLcited by 75
  9. 2021Few-Shot Bayesian Optimization with Deep Kernel Surrogatescited by 71
  10. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  11. 2024OmniPred: Language Models as Universal Regressorscited by 34
  12. 2018Regret bounds for meta Bayesian optimization with an unknown Gaussian process priorcited by 31
5 more
  1. 2023Transfer Learning for Bayesian Optimization: A Surveycited by 27
  2. 2021Transfer Learning with Gaussian Processes for Bayesian Optimizationcited by 26
  3. 2020Hyperparameter Transfer Across Developer Adjustmentscited by 11
  4. 2023Transfer Learning for Bayesian Optimization on Heterogeneous Search Spacescited by 9
  5. 2022On Provably Robust Meta-Bayesian Optimizationcited by 8

Built on by

Not cited yet.