Inertial Newton Algorithms Avoiding Strict Saddle Points
Not on the map. Why?
Abstract (arXiv)
We study the asymptotic behavior of second-order algorithms mixing Newton's method and inertial gradient descent in non-convex landscapes. We show that, despite the Newtonian behavior of these methods, they almost always escape strict saddle points. We also evidence the role played by the hyper-parameters of these methods in their qualitative behavior near critical points. The theoretical results are supported by numerical illustrations.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 21 of its 53 references that are in the map. First posted on arXiv 8 Nov 2021.
Builds on
21 of 53 references.
References · 21
- 2014Identifying and attacking the saddle point problem in high-dimensional non-convex optimizationcited by 286
- 2015A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insightscited by 203
- 2020On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problemscited by 79
- 2018Understanding the Acceleration Phenomenon via High-Resolution Differential Equationscited by 68
- 2019First-order methods almost always avoid saddle points: the case of vanishing step-sizescited by 35
- 2016Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regionscited by 33
- 2019First-order optimization algorithms via inertial systems with Hessian driven dampingcited by 22
- 2019An Inertial Newton Algorithm for Deep Learningcited by 14
- 2016Fast convex optimization via inertial dynamics with Hessian driven dampingcited by 13
- 2016Local Convergence of the Heavy-ball Method and iPiano for Non-convex Optimizationcited by 9
- 2017Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3cited by 9
- 2014An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functionscited by 8
9 more
- 2018Optimal convergence rates for Nesterov accelerationcited by 8
- 2017Behavior of Accelerated Gradient Methods Near Critical Points of Nonconvex Functionscited by 7
- 2018Backtracking gradient descent method for general C1 functions, with applications to Deep Learningcited by 6
- 2019First order optimization methods based on Hessian-driven Nesterov accelerated gradient flowcited by 6
- 2019An extension of the second order dynamical system that models Nesterov's convex gradient methodcited by 5
- 2020Fast optimization via inertial dynamics with closed-loop dampingcited by 4
- 2019Convergence to minima for the continuous version of Backtracking Gradient Descentcited by 3
- 2021Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven dampingcited by 3
- 2019Tikhonov regularization of a second order dynamical system with Hessian driven dampingcited by 1
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.