Can We Understand Plasticity Through Neural Collapse?
Not on the map. Why?
Abstract (arXiv)
This paper explores the connection between two recently identified phenomena in deep learning: plasticity loss and neural collapse. We analyze their correlation in different scenarios, revealing a significant association during the initial training phase on the first task. Additionally, we introduce a regularization approach to mitigate neural collapse, demonstrating its effectiveness in alleviating plasticity loss in this specific setting.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 8 of its 12 references that are in the map. First posted on arXiv 3 Apr 2024.
Builds on
8 of 12 references.
References · 8
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
- 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
- 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
- 2023Understanding plasticity in neural networkscited by 170
- 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
- 2020In Search of Robust Measures of Generalizationcited by 79
- 2017Critical Learning Periods in Deep Neural Networkscited by 78
- 2023Invariant Collaborative Filtering to Popularity Distribution Shiftcited by 23
Built on by
Citing papers per year: 2024: 2.
2 papers.