# Convergence to minima for the continuous version of Backtracking Gradient Descent

arXiv: https://arxiv.org/abs/1911.04221 · HTML: https://mlsanity.com/paper/1911.04221/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 3 of 3)

- [Inertial Newton Algorithms Avoiding Strict Saddle Points](https://mlsanity.com/paper/2111.04596/) (2021)
- [New Q-Newton's method meets Backtracking line search: good convergence guarantee, saddle points avoidance, quadratic rate of convergence, and easy implementation](https://mlsanity.com/paper/2108.10249/) (2021)
- [Generalisations and improvements of New Q-Newton's method Backtracking](https://mlsanity.com/paper/2109.11395/) (2021)

## Abstract (arXiv)

The main result of this paper is: bf Theorem. Let f:ℝkrightarrow ℝ be a C1 function, so that nabla f is locally Lipschitz continuous. Assume moreover that f is C2 near its generalised saddle points. Fix real numbers δ0>0 and 0<α <1. Then there is a smooth function h:ℝkrightarrow (0,δ0] so that the map H:ℝkrightarrow ℝk defined by H(x)=x-h(x)nabla f(x) has the following property: (i) For all x∈ ℝk, we have f(H(x)))-f(x)≤ -α h(x)||nabla f(x)||2. (ii) For every x0∈ ℝk, the sequence xn+1=H(xn) either satisfies limnrightarrow∞||xn+1-xn||=0 or  limnrightarrow∞||xn||=∞. Each cluster point of xn is a critical point of f. If moreover f has at most countably many critical points, then xn either converges to a critical point of f or limnrightarrow∞||xn||=∞. (iii) There is a set E1subset ℝk of Lebesgue measure 0 so that for all x0∈ ℝkbackslash E1, the sequence xn+1=H(xn), bf if converges, cannot converge to a bf generalised saddle point. (iv) There is a set E2subset ℝk of Lebesgue measure 0 so that for all x0∈ ℝkbackslash E2, any cluster point of the sequence xn+1=H(xn) is not a saddle point, and more generally cannot be an isolated generalised saddle point. Some other results are proven.