# Escaping saddle points without Lipschitz smoothness: the power of nonlinear preconditioning

arXiv: https://arxiv.org/abs/2509.15817 · HTML: https://mlsanity.com/paper/2509.15817/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2025.

## Built on by (most cited first, 3 of 3)

- [Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime](https://mlsanity.com/paper/2603.10485/) (2026)
- [Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives](https://mlsanity.com/paper/2605.11850/) (2026)
- [Deep Unfolded Latent Optimally Partitioned-l2/l1 Networks for Data-driven Block-Sparse Recovery](https://mlsanity.com/paper/2606.12740/) (2026)

## Abstract (arXiv)

We study generalized smoothness in nonconvex optimization, focusing on (L0, L1)-smoothness and anisotropic smoothness. The former was empirically derived from practical neural network training examples, while the latter arises naturally in the analysis of nonlinearly preconditioned gradient methods. We introduce a new sufficient condition that encompasses both notions, reveals their close connection, and holds in key applications such as phase retrieval and matrix factorization. Leveraging tools from dynamical systems theory, we then show that nonlinear preconditioning -- including gradient clipping -- preserves the saddle point avoidance property of classical gradient descent. Crucially, the assumptions required for this analysis are actually satisfied in these applications, unlike in classical results that rely on restrictive Lipschitz smoothness conditions. We further analyze a perturbed variant that efficiently attains second-order stationarity with only logarithmic dependence on dimension, matching similar guarantees of classical gradient methods.