Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting
Not on the map. Why?
Abstract (arXiv)
This paper investigates the sample dependence of critical points for neural networks. We introduce a sample-independent critical lifting operator that associates a parameter of one network with a set of parameters of another, thus defining sample-dependent and sample-independent lifted critical points. We then show by example that previously studied critical embeddings do not capture all sample-independent lifted critical points. Finally, we demonstrate the existence of sample-dependent lifted critical points for sufficiently large sample sizes and prove that saddles appear among them.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 20 references that are in the map. First posted on arXiv 19 May 2025.
Builds on
12 of 20 references.
References · 12
- 2016Deep Learning without Poor Local Minimacited by 234
- 2021Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariancescited by 127
- 2017The loss surface of deep and wide neural networkscited by 74
- 2020The Global Landscape of Neural Networks: An Overviewcited by 57
- 2019On Connected Sublevel Sets in Deep Learningcited by 51
- 2021Embedding Principle of Loss Landscape of Deep Neural Networkscited by 38
- 2019Semi-flat minima and saddle points by embedding neural networks to overparameterizationcited by 30
- 2021Embedding Principle: a hierarchical structure of loss landscape of deep neural networkscited by 22
- 2018On the Benefit of Width for Neural Networks: Disappearance of Bad Basinscited by 16
- 2017First-order Methods Almost Always Avoid Saddle Pointscited by 14
- 2023Should Under-parameterized Student Networks Copy or Average Teacher Weights?cited by 8
- 2022Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networkscited by 7
Built on by
Not cited yet.