mlsanityData to 14 Oct 2026

A Study of Neural Collapse Phenomenon: Grassmannian Frame, Symmetry and Generalization

arXiv 2304.08914 ↗First posted 18 Apr 2023cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

In this paper, we extend original Neural Collapse Phenomenon by proving Generalized Neural Collapse hypothesis. We obtain Grassmannian Frame structure from the optimization and generalization of classification. This structure maximally separates features of every two classes on a sphere and does not require a larger feature dimension than the number of classes. Out of curiosity about the symmetry of Grassmannian Frame, we conduct experiments to explore if models with different Grassmannian Frames have different performance. As a result, we discover the Symmetric Generalization phenomenon. We provide a theorem to explain Symmetric Generalization of permutation. However, the question of why different directions of features can lead to such different generalization is still open for future investigation.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 18 Apr 2023.

Builds on

25 of 44 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  5. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  6. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 320
  7. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 227
  8. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  9. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 158
  10. 2020Neural collapse with unconstrained featurescited by 142
  11. 2019Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Groupcited by 134
  12. 2023Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learningcited by 116
13 more
  1. 2022Extended Unconstrained Features Model for Exploring Deep Neural Collapsecited by 102
  2. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  3. 2022On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Featurescited by 97
  4. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  5. 2019Trivializations for Gradient-Based Optimization on Manifoldscited by 79
  6. 2022Are All Losses Created Equal: A Neural Collapse Perspectivecited by 78
  7. 2021Dissecting Supervised Contrastive Learningcited by 64
  8. 2022Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifoldcited by 50
  9. 2022Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learningcited by 43
  10. 2020Explicit regularization and implicit bias in deep network classifiers trained with the square losscited by 37
  11. 2021Learning with Multiclass AUC: Theory and Algorithmscited by 35
  12. 2023Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gapcited by 14
  13. 2015A study of the classification of low-dimensional data with supervised manifold learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 0; 2026: 1.

7 papers.

  1. 2023Generalized Neural Collapse for a Large Number of Classescited by 39
  2. 2024Linguistic Collapse: Neural Collapse in (Large) Language Modelscited by 38
  3. 2023Neural Collapse in Multi-label Learning with Pick-all-label Losscited by 14
  4. 2023Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batchingcited by 5
  5. 2023On the Dynamics Under the Unhinged Loss and Beyondcited by 3
  6. 2024Deep Neural Network Models Trained With A Fixed Random Classifier Transfer Better Across Domainscited by 2
  7. 2026Neural Collapse Dynamics: Depth, Activation, Regularisation, and Feature Norm Thresholdcited by 2