# Some convergent results for Backtracking Gradient Descent method on Banach spaces

arXiv: https://arxiv.org/abs/2001.05768 · HTML: https://mlsanity.com/paper/2001.05768/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2020.

## Built on by (most cited first, 3 of 3)

- [New Q-Newton's method meets Backtracking line search: good convergence guarantee, saddle points avoidance, quadratic rate of convergence, and easy implementation](https://mlsanity.com/paper/2108.10249/) (2021)
- [Generalisations and improvements of New Q-Newton's method Backtracking](https://mlsanity.com/paper/2109.11395/) (2021)
- [PEIRA: Learning Predictive Encoders through Inter-View Regressor Alignment](https://mlsanity.com/paper/2605.17671/) (2026)

## Abstract (arXiv)

Our main result concerns the following condition: bf Condition C. Let X be a Banach space. A C1 function f:Xrightarrow ℝ satisfies Condition C if whenever xn weakly converges to x and lim nrightarrow∞||nabla f(xn)||=0, then nabla f(x)=0. We assume that there is given a canonical isomorphism between X and its dual X^*, for example when X is a Hilbert space. bf Theorem. Let X be a reflexive, complete Banach space and f:Xrightarrow ℝ be a C2 function which satisfies Condition C. Moreover, we assume that for every bounded set Ssubset X, then sup x∈ S||nabla 2f(x)||<∞. We choose a random point x0∈ X and construct by the Local Backtracking GD procedure (which depends on 3 hyper-parameters α ,β ,δ 0, see later for details) the sequence xn+1=xn-δ (xn)nabla f(xn). Then we have: 1) Every cluster point of xn, in the bf weak topology, is a critical point of f. 2) Either lim nrightarrow∞f(xn)=-∞ or lim nrightarrow∞||xn+1-xn||=0. 3) Here we work with the weak topology. Let C be the set of critical points of f. Assume that C has a bounded component A. Let B be the set of cluster points of xn. If Bcap Anot= emptyset, then Bsubset A and B is connected. 4) Assume that X is separable. Then for generic choices of α ,β ,δ 0 and the initial point x0, if the sequence xn converges - in the bf weak topology, then the limit point cannot be a saddle point.