# A Derivative-Free Gauss-Newton Method

arXiv: https://arxiv.org/abs/1710.11005 · HTML: https://mlsanity.com/paper/1710.11005/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2017.

## Built on by (most cited first, 3 of 3)

- [An adaptively inexact first-order method for bilevel optimization with application to hyperparameter learning](https://mlsanity.com/paper/2308.10098/) (2023)
- [Ensemble-based gradient inference for particle methods in optimization and sampling](https://mlsanity.com/paper/2209.15420/) (2022)
- [Agentic TCAD Calibration Workflow for Oxide Semiconductor Transistors](https://mlsanity.com/paper/2609.12184/) (2026)

## Abstract (arXiv)

We present DFO-GN, a derivative-free version of the Gauss-Newton method for solving nonlinear least-squares problems. As is common in derivative-free optimization, DFO-GN uses interpolation of function values to build a model of the objective, which is then used within a trust-region framework to give a globally-convergent algorithm requiring O(ε-2) iterations to reach approximate first-order criticality within tolerance ε. This algorithm is a simplification of the method from [H. Zhang, A. R. Conn, and K. Scheinberg, A Derivative-Free Algorithm for Least-Squares Minimization, SIAM J. Optim., 20 (2010), pp. 3555-3576], where we replace quadratic models for each residual with linear models. We demonstrate that DFO-GN performs comparably to the method of Zhang et al. in terms of objective evaluations, as well as having a substantially faster runtime and improved scalability.