# Recent Advances in Non-convex Smoothness Conditions and Applicability to Deep Linear Neural Networks

arXiv: https://arxiv.org/abs/2409.13672 · HTML: https://mlsanity.com/paper/2409.13672/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 12 of 12)

- [Why gradient clipping accelerates training: A theoretical justification for adaptivity](https://mlsanity.com/paper/1905.11881/) (2019)
- [Improved Analysis of Clipping Algorithms for Non-convex Optimization](https://mlsanity.com/paper/2010.02519/) (2020)
- [Adaptive Gradient Descent without Descent](https://mlsanity.com/paper/1910.09529/) (2019)
- [Convex and Non-convex Optimization Under Generalized Smoothness](https://mlsanity.com/paper/2306.01264/) (2023)
- [Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization](https://mlsanity.com/paper/2303.02854/) (2023)
- [WNGrad: Learn the Learning Rate in Gradient Descent](https://mlsanity.com/paper/1803.02865/) (2018)
- [Global Convergence and Stability of Stochastic Gradient Descent](https://mlsanity.com/paper/2110.01663/) (2021)
- [Gradient Descent in the Absence of Global Lipschitz Continuity of the Gradients](https://mlsanity.com/paper/2210.02418/) (2022)
- [Stabilized Barzilai-Borwein method](https://mlsanity.com/paper/1907.06409/) (2019)
- [Exploiting Negative Curvature in Deterministic and Stochastic Optimization](https://mlsanity.com/paper/1703.00412/) (2017)
- [A Novel Gradient Methodology with Economical Objective Function Evaluations for Data Science Applications](https://mlsanity.com/paper/2309.10894/) (2023)
- [The Challenges of Optimization For Data Science](https://mlsanity.com/paper/2404.09810/) (2024)

## Abstract (arXiv)

The presence of non-convexity in smooth optimization problems arising from deep learning have sparked new smoothness conditions in the literature and corresponding convergence analyses. We discuss these smoothness conditions, order them, provide conditions for determining whether they hold, and evaluate their applicability to training a deep linear neural network for binary classification.