mlsanityData to 14 Oct 2026

Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learning

arXiv 2204.08735 ↗First posted 19 Apr 2022cs.LG stat.MLcited by 43

Not on the map. Why?

Abstract (arXiv)

Class imbalance distribution widely exists in real-world engineering. However, the mainstream optimization algorithms that seek to minimize error will trap the deep learning model in sub-optimums when facing extreme class imbalance. It seriously harms the classification precision, especially on the minor classes. The essential reason is that the gradients of the classifier weights are imbalanced among the components from different classes. In this paper, we propose Attraction-Repulsion-Balanced Loss (ARB-Loss) to balance the different components of the gradients. We perform experiments on the large-scale classification and segmentation datasets and our ARB-Loss can achieve state-of-the-art performance via only one-stage training instead of 2-stage learning like nowadays SOTA works.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 19 Apr 2022.

Builds on

29 of 45 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  8. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  9. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  10. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  11. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  12. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 705
17 more
  1. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 513
  2. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  3. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 400
  4. 2020Calibrating Deep Neural Networks using Focal Losscited by 357
  5. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  6. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 227
  7. 2021Improving Calibration for Long-Tailed Recognitioncited by 212
  8. 2020Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classificationcited by 175
  9. 2015Cost Sensitive Learning of Deep Feature Representations from Imbalanced Datacited by 167
  10. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  11. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 158
  12. 2020Remix: Rebalanced Mixupcited by 144
  13. 2020Neural collapse with unconstrained featurescited by 142
  14. 2016Range Loss for Deep Face Recognition with Long-tailcited by 111
  15. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  16. 2020Neural Collapse with Cross-Entropy Losscited by 52
  17. 2017Loss Max-Pooling for Semantic Image Segmentationcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 18; 2024: 8; 2025: 10; 2026: 3.

43 papers.

Sort:

By area:

  1. 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Modelscited by 119
  2. 2023Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learningcited by 116
  3. 2022Imbalance Trouble: Revisiting Neural-Collapse Geometrycited by 80
  4. 2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
  5. 2022Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifoldcited by 50
  6. 2023Understanding Imbalanced Semantic Segmentation Through Neural Collapsecited by 46
  7. 2023Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Modelcited by 43
  8. 2023Generalized Neural Collapse for a Large Number of Classescited by 39
  9. 2023Inducing Neural Collapse in Deep Long-tailed Learningcited by 28
  10. 2022Understanding and Improving Transfer Learning of Deep Models via Neural Collapsecited by 24
  11. 2023On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Datacited by 23
  12. 2023Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variantscited by 14
31 more
  1. 2023Neural Collapse in Multi-label Learning with Pick-all-label Losscited by 14
  2. 2022Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learningcited by 13
  3. 2024Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapsecited by 13
  4. 2023Fine-tune Language Models to Approximate Unbiased In-context Learningcited by 12
  5. 2024Progressive Feedforward Collapse of ResNet Trainingcited by 12
  6. 2024Cross Entropy versus Label Smoothing: A Neural Collapse Perspectivecited by 11
  7. 2023A Study of Neural Collapse Phenomenon: Grassmannian Frame, Symmetry and Generalizationcited by 7
  8. 2023Towards Demystifying the Generalization Behaviors When Neural Collapse Emergescited by 7
  9. 2024Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretizationcited by 7
  10. 2024Adaptive Parametric Activation: Unifying and Generalising Activation Functions Across Taskscited by 6
  11. 2023Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batchingcited by 5
  12. 2024Supervised Contrastive Representation Learning: Landscape Analysis with Unconstrained Featurescited by 5
  13. 2023Neural Collapse Inspired Federated Learning with Non-iid Datacited by 4
  14. 2025BatStyler: Advancing Multi-category Style Generation for Source-free Domain Generalizationcited by 4
  15. 2025Rethinking Continual Learning with Progressive Neural Collapsecited by 4
  16. 2023Long-Tail Learning with Rebalanced Contrastive Losscited by 3
  17. 2024Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspectivecited by 3
  18. 2025Balancing Two Classifiers via A Simplex ETF Structure for Model Calibrationcited by 3
  19. 2023Understanding Prompt Tuning for V-L Models Through the Lens of Neural Collapsecited by 2
  20. 2025Space Alignment Matters: The Missing Piece for Inducing Neural Collapse in Long-Tailed Learningcited by 2
  21. 2026Neural Collapse Dynamics: Depth, Activation, Regularisation, and Feature Norm Thresholdcited by 2
  22. 2023Few-shot 1/a Anomalies Feedback : Damage Vision Mining Opportunity and Embedding Feature Imbalancecited by 1
  23. 2024Evaluating the Fairness of Neural Collapse in Medical Image Classificationcited by 1
  24. 2025From Coefficients to Directions: Rethinking Model Merging with Directional Alignmentcited by 1
  25. 2025Enhancing Pre-Trained Model-Based Class-Incremental Learning through Neural Collapsecited by 0
  26. 2025rETF-semiSL: Semi-Supervised Learning for Neural Collapse in Temporal Datacited by 0
  27. 2025Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradientcited by 0
  28. 2025Exploring "Many in Few" and "Few in Many" Properties in Long-Tailed, Highly-Imbalanced IC Defect Classificationcited by 0
  29. 20253D-ANC: Adaptive Neural Collapse for Robust 3D Point Cloud Recognitioncited by 0
  30. 2026Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of Viewcited by 0
  31. 2026From Latent Space to Training Data: Explainable Specialization in Minimal MLPscited by 0