# Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods

arXiv: https://arxiv.org/abs/1803.10173 · HTML: https://mlsanity.com/paper/1803.10173/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2018.

## Built on by (most cited first, 7 of 7)

- [Re-parameterizing Your Optimizers rather than Architectures](https://mlsanity.com/paper/2205.15242/) (2022)
- [A Sequential Quadratic Programming Method with High Probability Complexity Bounds for Nonlinear Equality Constrained Stochastic Optimization](https://mlsanity.com/paper/2301.00477/) (2023)
- [Adaptive Sampling Quasi-Newton Methods for Zeroth-Order Stochastic Optimization](https://mlsanity.com/paper/2109.12213/) (2021)
- [A Principle for Global Optimization with Gradients](https://mlsanity.com/paper/2308.09556/) (2023)
- [The limitation of neural nets for approximation and optimization](https://mlsanity.com/paper/2311.12253/) (2023)
- [Derivative-Free Optimization via Finite Difference Approximation: An Experimental Study](https://mlsanity.com/paper/2411.00112/) (2024)
- [Enhanced Derivative-Free Optimization Using Adaptive Correlation-Induced Finite Difference Estimators](https://mlsanity.com/paper/2502.20819/) (2025)

## Abstract (arXiv)

This paper presents a finite difference quasi-Newton method for the minimization of noisy functions. The method takes advantage of the scalability and power of BFGS updating, and employs an adaptive procedure for choosing the differencing interval h based on the noise estimation techniques of Hamming (2012) and Mor'e and Wild (2011). This noise estimation procedure and the selection of h are inexpensive but not always accurate, and to prevent failures the algorithm incorporates a recovery mechanism that takes appropriate action in the case when the line search procedure is unable to produce an acceptable point. A novel convergence analysis is presented that considers the effect of a noisy line search procedure. Numerical experiments comparing the method to a function interpolating trust region method are presented.