Manipulating Sparse Double Descent
Not on the map. Why?
Abstract (arXiv)
This paper investigates the double descent phenomenon in two-layer neural networks, focusing on the role of L1 regularization and representation dimensions. It explores an alternative double descent phenomenon, named sparse double descent. The study emphasizes the complex relationship between model complexity, sparsity, and generalization, and suggests further research into more diverse models and datasets. The findings contribute to a deeper understanding of neural network training and optimization.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 11 references that are in the map. First posted on arXiv 19 Jan 2024.
Builds on
6 of 11 references.
References · 6
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
- 2021Neural Networks as Kernel Learners: The Silent Alignment Effectcited by 106
- 2023A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learningcited by 33
- 2022Sparse Double Descent: Where Network Pruning Aggravates Overfittingcited by 26
Built on by
Not cited yet.