# Inertial Newton Algorithms Avoiding Strict Saddle Points

arXiv: https://arxiv.org/abs/2111.04596 · HTML: https://mlsanity.com/paper/2111.04596/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 53 references that are in the map.
- First posted on arXiv 8 Nov 2021.

## Builds on (most cited first, 21 of 21)

- [Identifying and attacking the saddle point problem in high-dimensional non-convex optimization](https://mlsanity.com/paper/1406.2572/) (2014)
- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems](https://mlsanity.com/paper/2006.11144/) (2020)
- [Understanding the Acceleration Phenomenon via High-Resolution Differential Equations](https://mlsanity.com/paper/1810.08907/) (2018)
- [First-order methods almost always avoid saddle points: the case of vanishing step-sizes](https://mlsanity.com/paper/1906.07772/) (2019)
- [Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions](https://mlsanity.com/paper/1605.00405/) (2016)
- [First-order optimization algorithms via inertial systems with Hessian driven damping](https://mlsanity.com/paper/1907.10536/) (2019)
- [An Inertial Newton Algorithm for Deep Learning](https://mlsanity.com/paper/1905.12278/) (2019)
- [Fast convex optimization via inertial dynamics with Hessian driven damping](https://mlsanity.com/paper/1601.07113/) (2016)
- [Local Convergence of the Heavy-ball Method and iPiano for Non-convex Optimization](https://mlsanity.com/paper/1606.09070/) (2016)
- [Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3](https://mlsanity.com/paper/1706.05671/) (2017)
- [An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functions](https://mlsanity.com/paper/1410.0641/) (2014)
- [Optimal convergence rates for Nesterov acceleration](https://mlsanity.com/paper/1805.05719/) (2018)
- [Behavior of Accelerated Gradient Methods Near Critical Points of Nonconvex Functions](https://mlsanity.com/paper/1706.07993/) (2017)
- [Backtracking gradient descent method for general C1 functions, with applications to Deep Learning](https://mlsanity.com/paper/1808.05160/) (2018)
- [First order optimization methods based on Hessian-driven Nesterov accelerated gradient flow](https://mlsanity.com/paper/1912.09276/) (2019)
- [An extension of the second order dynamical system that models Nesterov's convex gradient method](https://mlsanity.com/paper/1908.02574/) (2019)
- [Fast optimization via inertial dynamics with closed-loop damping](https://mlsanity.com/paper/2008.02261/) (2020)
- [Convergence to minima for the continuous version of Backtracking Gradient Descent](https://mlsanity.com/paper/1911.04221/) (2019)
- [Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven damping](https://mlsanity.com/paper/2107.05943/) (2021)
- [Tikhonov regularization of a second order dynamical system with Hessian driven damping](https://mlsanity.com/paper/1911.12845/) (2019)

## Built on by (most cited first, 2 of 2)

- [Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality](https://mlsanity.com/paper/2410.16849/) (2024)
- [Convergence of optimizers implies eigenvalues filtering at equilibrium](https://mlsanity.com/paper/2510.09034/) (2025)

## Abstract (arXiv)

We study the asymptotic behavior of second-order algorithms mixing Newton's method and inertial gradient descent in non-convex landscapes. We show that, despite the Newtonian behavior of these methods, they almost always escape strict saddle points. We also evidence the role played by the hyper-parameters of these methods in their qualitative behavior near critical points. The theoretical results are supported by numerical illustrations.