Recent Advances in Non-convex Smoothness Conditions and Applicability to Deep Linear Neural Networks
Not on the map. Why?
Abstract (arXiv)
The presence of non-convexity in smooth optimization problems arising from deep learning have sparked new smoothness conditions in the literature and corresponding convergence analyses. We discuss these smoothness conditions, order them, provide conditions for determining whether they hold, and evaluate their applicability to training a deep linear neural network for binary classification.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 20 Sep 2024.
Builds on
12 of 22 references.
References · 12
- 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
- 2020Improved Analysis of Clipping Algorithms for Non-convex Optimizationcited by 88
- 2019Adaptive Gradient Descent without Descentcited by 79
- 2023Convex and Non-convex Optimization Under Generalized Smoothnesscited by 44
- 2023Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimizationcited by 35
- 2018WNGrad: Learn the Learning Rate in Gradient Descentcited by 24
- 2021Global Convergence and Stability of Stochastic Gradient Descentcited by 17
- 2022Gradient Descent in the Absence of Global Lipschitz Continuity of the Gradientscited by 11
- 2019Stabilized Barzilai-Borwein methodcited by 10
- 2017Exploiting Negative Curvature in Deterministic and Stochastic Optimizationcited by 7
- 2023A Novel Gradient Methodology with Economical Objective Function Evaluations for Data Science Applicationscited by 1
- 2024The Challenges of Optimization For Data Sciencecited by 1
Built on by
Not cited yet.