# Optimization, fast and slow: optimally switching between local and Bayesian optimization

arXiv: https://arxiv.org/abs/1805.08610 · HTML: https://mlsanity.com/paper/1805.08610/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2018.

## Built on by (most cited first, 22 of 22)

- [Unexpected Improvements to Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2310.20708/) (2023)
- [Local policy search with Bayesian optimization](https://mlsanity.com/paper/2106.11899/) (2021)
- [Local Bayesian optimization via maximizing probability of descent](https://mlsanity.com/paper/2210.11662/) (2022)
- [TREGO: a Trust-Region Framework for Efficient Global Optimization](https://mlsanity.com/paper/2101.06808/) (2021)
- [Automatic Termination for Hyperparameter Optimization](https://mlsanity.com/paper/2104.08166/) (2021)
- [Stopping Bayesian Optimization with Probabilistic Regret Bounds](https://mlsanity.com/paper/2402.16811/) (2024)
- [Revisiting Bayesian Optimization in the light of the COCO benchmark](https://mlsanity.com/paper/2103.16649/) (2021)
- [User Preference Meets Pareto-Optimality in Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2502.06971/) (2025)
- [Predicting the utility of search spaces for black-box optimization: a simple, budget-aware approach](https://mlsanity.com/paper/2112.08250/) (2021)
- [Bayesian Optimization with Informative Covariance](https://mlsanity.com/paper/2208.02704/) (2022)
- [LABCAT: Locally adaptive Bayesian optimization using principal-component-aligned trust regions](https://mlsanity.com/paper/2311.11328/) (2023)
- [Hyper-parameter Tuning for Adversarially Robust Models](https://mlsanity.com/paper/2304.02497/) (2023)
- [HomOpt: A Homotopy-Based Hyperparameter Optimization Method](https://mlsanity.com/paper/2308.03317/) (2023)
- [Bayesian autoregression to optimize temporal Mat'ern kernel Gaussian process hyperparameters](https://mlsanity.com/paper/2508.09792/) (2025)
- [CobBO: Coordinate Backoff Bayesian Optimization with Two-Stage Kernels](https://mlsanity.com/paper/2101.05147/) (2021)
- [Searching in the Forest for Local Bayesian Optimization](https://mlsanity.com/paper/2111.05834/) (2021)
- [Dependency Structure Search Bayesian Optimization for Decision Making Models](https://mlsanity.com/paper/2308.00629/) (2023)
- [Adaptive Replication Strategies in Trust-Region-Based Bayesian Optimization of Stochastic Functions](https://mlsanity.com/paper/2504.20527/) (2025)
- [Efficient Sliced Wasserstein Distance Computation via Adaptive Bayesian Optimization](https://mlsanity.com/paper/2509.17405/) (2025)
- [LAGO: A Local-Global Optimization Framework Combining Trust Region Methods and Bayesian Optimization](https://mlsanity.com/paper/2603.02970/) (2026)
- [Regret-Based (ε,δ)-optimal Stopping Criteria for Bayesian Optimization](https://mlsanity.com/paper/2605.22561/) (2026)
- [GRAPE: Gradient Refinement and Progress-Aware Exploitation for Query-Efficient High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2608.25116/) (2026)

## Abstract (arXiv)

We develop the first Bayesian Optimization algorithm, BLOSSOM, which selects between multiple alternative acquisition functions and traditional local optimization at each step. This is combined with a novel stopping condition based on expected regret. This pairing allows us to obtain the best characteristics of both local and Bayesian optimization, making efficient use of function evaluations while yielding superior convergence to the global minimum on a selection of optimization problems, and also halting optimization once a principled and intuitive stopping condition has been fulfilled.