# Warm Starting Bayesian Optimization

arXiv: https://arxiv.org/abs/1608.03585 · HTML: https://mlsanity.com/paper/1608.03585/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2016.

## Built on by (most cited first, 21 of 21)

- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [Towards Learning Universal Hyperparameter Optimizers with Transformers](https://mlsanity.com/paper/2205.13320/) (2022)
- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [Transfer Learning for Bayesian Optimization: A Survey](https://mlsanity.com/paper/2302.05927/) (2023)
- [Sequential Large Language Model-Based Hyper-parameter Optimization](https://mlsanity.com/paper/2410.20302/) (2024)
- [Language-Based Bayesian Optimization Research Assistant (BORA)](https://mlsanity.com/paper/2501.16224/) (2025)
- [Reinforced In-Context Black-Box Optimization](https://mlsanity.com/paper/2402.17423/) (2024)
- [JUMBO: Scalable Multi-task Bayesian Optimization using Offline Data](https://mlsanity.com/paper/2106.00942/) (2021)
- [On Provably Robust Meta-Bayesian Optimization](https://mlsanity.com/paper/2206.06872/) (2022)
- [Pre-training helps Bayesian optimization too](https://mlsanity.com/paper/2207.03084/) (2022)
- [Improving Hyperparameter Optimization with Checkpointed Model Weights](https://mlsanity.com/paper/2406.18630/) (2024)
- [ADMIRE-BayesOpt: Accelerated Data MIxture RE-weighting for Language Models with Bayesian Optimization](https://mlsanity.com/paper/2508.11551/) (2025)
- [Amortized Auto-Tuning: Cost-Efficient Bayesian Transfer Optimization for Hyperparameter Recommendation](https://mlsanity.com/paper/2106.09179/) (2021)
- [None To Optima in Few Shots: Bayesian Optimization with MDP Priors](https://mlsanity.com/paper/2511.01006/) (2025)
- [Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery](https://mlsanity.com/paper/2605.22613/) (2026)
- [Robust Transfer Learning for Active Level Set Estimation with Locally Adaptive Gaussian Process Prior](https://mlsanity.com/paper/2410.05660/) (2024)
- [Provable Accelerated Bayesian Optimization with Knowledge Transfer](https://mlsanity.com/paper/2511.03125/) (2025)
- [Why risk matters for protein binder design](https://mlsanity.com/paper/2504.00146/) (2025)
- [An Empirical Study on Ensemble-Based Transfer Learning Bayesian Optimisation with Mixed Variable Types](https://mlsanity.com/paper/2601.15640/) (2026)
- [SMOG: Scalable Meta-Learning for Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2601.22131/) (2026)
- [Harnessing Large Language Models to Compile Task-Relevant Context into Bayesian Optimisation](https://mlsanity.com/paper/2609.36788/) (2026)

## Abstract (arXiv)

We develop a framework for warm-starting Bayesian optimization, that reduces the solution time required to solve an optimization problem that is one in a sequence of related problems. This is useful when optimizing the output of a stochastic simulator that fails to provide derivative information, for which Bayesian optimization methods are well-suited. Solving sequences of related optimization problems arises when making several business decisions using one optimization model and input data collected over different time periods or markets. While many gradient-based methods can be warm started by initiating optimization at the solution to the previous problem, this warm start approach does not apply to Bayesian optimization methods, which carry a full metamodel of the objective function from iteration to iteration. Our approach builds a joint statistical model of the entire collection of related objective functions, and uses a value of information calculation to recommend points to evaluate.