# On non-approximability of zero loss global mathcal L2 minimizers by gradient descent in Deep Learning

arXiv: https://arxiv.org/abs/2311.07065 · HTML: https://mlsanity.com/paper/2311.07065/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 11 references that are in the map.
- First posted on arXiv 13 Nov 2023.

## Builds on (most cited first, 7 of 7)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [How to Start Training: The Effect of Initialization and Architecture](https://mlsanity.com/paper/1803.01719/) (2018)
- [Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions](https://mlsanity.com/paper/2006.15459/) (2020)
- [Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape](https://mlsanity.com/paper/2305.19510/) (2023)
- [Geometric structure of Deep Learning networks and construction of global mathcal L2 minimizers](https://mlsanity.com/paper/2309.10639/) (2023)
- [Geometric structure of shallow neural networks and constructive mathcal L2 cost minimization](https://mlsanity.com/paper/2309.10370/) (2023)

## Abstract (arXiv)

We analyze geometric aspects of the gradient descent algorithm in Deep Learning (DL), and give a detailed discussion of the circumstance that in underparametrized DL networks, zero loss minimization can generically not be attained. As a consequence, we conclude that the distribution of training inputs must necessarily be non-generic in order to produce zero loss minimizers, both for the method constructed in [Chen-Munoz Ewald 2023, 2024], or for gradient descent [Chen 2025] (which assume clustering of training data).