mlsanityData to 7 Oct 2026

All-around Neural Collapse for Imbalanced Classification

arXiv 2408.07253 ↗First posted 14 Aug 2024cs.LG cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Neural Collapse (NC) presents an elegant geometric structure that enables individual activations (features), class means and classifier (weights) vectors to reach textitoptimal inter-class separability during the terminal phase of training on a textitbalanced dataset. Once shifted to imbalanced classification, such an optimal structure of NC can be readily destroyed by the notorious textitminority collapse, where the classifier vectors corresponding to the minority classes are squeezed. In response, existing works endeavor to recover NC typically by optimizing classifiers. However, we discover that this squeezing phenomenon is not only confined to classifier vectors but also occurs with class means. Consequently, reconstructing NC solely at the classifier aspect may be futile, as the feature means remain compressed, leading to the violation of inherent textitself-duality in NC (textiti.e., class means and classifier vectors converge mutually) and incidentally, resulting in an unsatisfactory collapse of individual activations towards the corresponding class means. To shake off these dilemmas, we present a unified All-around Neural Collapse framework (AllNC), aiming to comprehensively restore NC across multiple aspects including individual activations, class means and classifier vectors. We thoroughly analyze its effectiveness and verify on multiple benchmark datasets that it achieves state-of-the-art in both balanced and imbalanced settings.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 14 Aug 2024.

Builds on

40 of 53 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  4. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  5. 2020Supervised Contrastive Learningcited by 2,803
  6. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  7. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  8. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  9. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  10. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
  11. 2021Deep Long-Tailed Learning: A Surveycited by 416
  12. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
28 more
  1. 2021Parametric Contrastive Learningcited by 217
  2. 2021Improving Calibration for Long-Tailed Recognitioncited by 212
  3. 2021Distribution Alignment: A Unified Framework for Long-tail Visual Recognitioncited by 198
  4. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  5. 2022Balanced Contrastive Learning for Long-Tailed Visual Recognitioncited by 143
  6. 2021Targeted Supervised Contrastive Learning for Long-Tailed Recognitioncited by 132
  7. 2015Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networkscited by 115
  8. 2022Long-Tailed Recognition via Weight Balancingcited by 107
  9. 2022Extended Unconstrained Features Model for Exploring Deep Neural Collapsecited by 101
  10. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  11. 2022Imbalance Trouble: Revisiting Neural-Collapse Geometrycited by 80
  12. 2017Deep Over-sampling Framework for Classifying Imbalanced Datacited by 58
  13. 2022Equalized Focal Loss for Dense Long-Tailed Object Detectioncited by 46
  14. 2023Global and Local Mixture Consistency Cumulative Learning for Long-tailed Visual Recognitionscited by 43
  15. 2023Generalized Neural Collapse for a Large Number of Classescited by 39
  16. 2023How Re-sampling Helps for Long-Tail Learning?cited by 36
  17. 2023Inducing Neural Collapse in Deep Long-tailed Learningcited by 28
  18. 2023Subclass-balancing Contrastive Learning for Long-tailed Recognitioncited by 26
  19. 2021Imagine by Reasoning: A Reasoning-Based Implicit Semantic Data Augmentation for Long-Tailed Classificationcited by 24
  20. 2022Maximum Class Separation as Inductive Bias in One Matrixcited by 21
  21. 2023Neural Collapse in the Intermediate Hidden Layers of Classification Neural Networkscited by 20
  22. 2022ResNorm: Tackling Long-tailed Degree Distribution Issue in Graph Neural Networks via Normalizationcited by 15
  23. 2024Learning Equi-angular Representations for Online Continual Learningcited by 15
  24. 2024Neighborhood-Enhanced Supervised Contrastive Learning for Collaborative Filteringcited by 10
  25. 2020Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extractioncited by 8
  26. 2023Learning Prototype Classifiers for Long-Tailed Recognitioncited by 6
  27. 2022Rebalanced Siamese Contrastive Mining for Long-Tailed Recognitioncited by 4
  28. 2024Hyperspherical Classification with Dynamic Label-to-Prototype Assignmentcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Contrastive-Collapsed Loss for Flexible and Geometrically Optimal Embeddings and Faster Convergencecited by 0
  2. 2026Information Density Imbalance in Visual Object Detectioncited by 0
  3. 2026Rethinking Class Imbalance for Single-Cell Foundation Models: A Systematic Benchmark Across Architectures and Long-Tail Loss Functionscited by 0