mlsanityData to 7 Oct 2026

Neural Collapse with Cross-Entropy Loss

arXiv 2012.08465 ↗First posted 15 Dec 2020cs.LG math.CAcited by 52

Not on the map. Why?

Abstract (arXiv)

We consider the variational problem of cross-entropy loss with n feature vectors on a unit hypersphere in ℝd. We prove that when d ≥ n - 1, the global minimum is given by the simplex equiangular tight frame, which justifies the neural collapse behavior. We also prove that as n rightarrow ∞ with fixed d, the minimizing points will distribute uniformly on the hypersphere and show a connection with the frame potential of Benedetto & Fickus.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 13; 2023: 18; 2024: 11; 2025: 2; 2026: 3.

52 papers.

Sort:

By area:

  1. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
  2. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  3. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
  4. 2023Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learningcited by 112
  5. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  6. 2022On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Featurescited by 97
  7. 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
  8. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  9. 2022Imbalance Trouble: Revisiting Neural-Collapse Geometrycited by 80
  10. 2022Are All Losses Created Equal: A Neural Collapse Perspectivecited by 78
  11. 2021Label-Imbalanced and Group-Sensitive Classification under Overparameterizationcited by 74
  12. 2022Limitations of Neural Collapse for Understanding Generalization in Deep Learningcited by 63
38 more
  1. 2022Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifoldcited by 50
  2. 2023Understanding Imbalanced Semantic Segmentation Through Neural Collapsecited by 46
  3. 2022Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learningcited by 43
  4. 2022Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learningcited by 40
  5. 2023Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced Datacited by 40
  6. 2023Generalized Neural Collapse for a Large Number of Classescited by 39
  7. 2024Linguistic Collapse: Neural Collapse in (Large) Language Modelscited by 36
  8. 2023Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representationscited by 31
  9. 2022Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networkscited by 23
  10. 2023On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Datacited by 23
  11. 2022Nearest Class-Center Simplification through Intermediate Layerscited by 20
  12. 2023Neural Collapse in the Intermediate Hidden Layers of Classification Neural Networkscited by 20
  13. 2024Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Modelcited by 19
  14. 2024Learning Equi-angular Representations for Online Continual Learningcited by 15
  15. 2023Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variantscited by 14
  16. 2022On the Implicit Bias Towards Minimal Depth of Deep Neural Networkscited by 12
  17. 2024Progressive Feedforward Collapse of ResNet Trainingcited by 12
  18. 2023Are Neurons Actually Collapsed? On the Fine-Grained Structure in Neural Representationscited by 10
  19. 2023Quantifying the Variability Collapse of Neural Networkscited by 9
  20. 2023Mini-Batch Optimization of Contrastive Losscited by 8
  21. 2024Memory-efficient Continual Learning with Neural Collapse Contrastivecited by 8
  22. 2023Exploring Weight Balancing on Long-Tailed Recognition Problemcited by 7
  23. 2024Pushing Boundaries: Mixup's Influence on Neural Collapsecited by 7
  24. 2023Feature Collapsecited by 6
  25. 2023Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batchingcited by 5
  26. 2024Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapsecited by 5
  27. 2024Supervised Contrastive Representation Learning: Landscape Analysis with Unconstrained Featurescited by 5
  28. 2023Neural Collapse Inspired Federated Learning with Non-iid Datacited by 4
  29. 2023Exploring Simple, High Quality Out-of-Distribution Detection with L2 Normalizationcited by 3
  30. 2023DRAUC: An Instance-wise Distributionally Robust AUC Optimization Frameworkcited by 3
  31. 2024Hyperspherical Classification with Dynamic Label-to-Prototype Assignmentcited by 3
  32. 2022Memorization-Dilation: Modeling Neural Collapse Under Label Noisecited by 2
  33. 2023An Investigation of Representation and Allocation Harms in Contrastive Learningcited by 2
  34. 2024Deep Neural Network Models Trained With A Fixed Random Classifier Transfer Better Across Domainscited by 2
  35. 2026On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvaturecited by 2
  36. 2024Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustmentcited by 1
  37. 2025SAMix: Calibrated and Accurate Continual Learning via Sphere-Adaptive Mixup and Neural Collapsecited by 1
  38. 2026Safety-Efficacy Trade Off: Robustness against Data-Poisoningcited by 1

50 most cited of 52 All 52 (JSON)