# Escaping local minima with derivative-free methods: a numerical investigation

arXiv: https://arxiv.org/abs/1812.11343 · HTML: https://mlsanity.com/paper/1812.11343/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Dec 2018.

## Built on by (most cited first, 4 of 4)

- [Deterministic Global Optimization of the Acquisition Function in Bayesian Optimization: To Do or Not To Do?](https://mlsanity.com/paper/2503.03625/) (2025)
- [Exploiting Function-Family Structure in Analog Circuit Optimization](https://mlsanity.com/paper/2512.00712/) (2025)
- [Geoff: The Generic Optimization Framework & Frontend for Particle Accelerator Controls](https://mlsanity.com/paper/2506.03796/) (2025)
- [Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization](https://mlsanity.com/paper/2510.15435/) (2025)

## Abstract (arXiv)

We apply a state-of-the-art, local derivative-free solver, Py-BOBYQA, to global optimization problems, and propose an algorithmic improvement that is beneficial in this context. Our numerical findings are illustrated on a commonly-used but small-scale test set of global optimization problems and associated noisy variants, and on hyperparameter tuning for the machine learning test set MNIST. As Py-BOBYQA is a model-based trust-region method, we compare mostly (but not exclusively) with other global optimization methods for which (global) models are important, such as Bayesian optimization and response surface methods; we also consider state-of-the-art representative deterministic and stochastic codes, such as DIRECT and CMA-ES. As a heuristic for escaping local minima, we find numerically that Py-BOBYQA is competitive with global optimization solvers for all accuracy/budget regimes, in both smooth and noisy settings. In particular, Py-BOBYQA variants are best performing for smooth and multiplicative noise problems in high-accuracy regimes. As a by-product, some preliminary conclusions can be drawn on the relative performance of the global solvers we have tested with default settings.