On non-approximability of zero loss global mathcal L2 minimizers by gradient descent in Deep Learning
Not on the map. Why?
Abstract (arXiv)
We analyze geometric aspects of the gradient descent algorithm in Deep Learning (DL), and give a detailed discussion of the circumstance that in underparametrized DL networks, zero loss minimization can generically not be attained. As a consequence, we conclude that the distribution of training inputs must necessarily be non-generic in order to produce zero loss minimizers, both for the method constructed in [Chen-Munoz Ewald 2023, 2024], or for gradient descent [Chen 2025] (which assume clustering of training data).
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 11 references that are in the map. First posted on arXiv 13 Nov 2023.
Builds on
7 of 11 references.
References · 7
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
- 2018How to Start Training: The Effect of Initialization and Architecturecited by 109
- 2020Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functionscited by 50
- 2023Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscapecited by 17
- 2023Geometric structure of Deep Learning networks and construction of global mathcal L2 minimizerscited by 6
- 2023Geometric structure of shallow neural networks and constructive mathcal L2 cost minimizationcited by 4
Built on by
Not cited yet.