# Inducing Neural Collapse in Deep Long-tailed Learning

arXiv: https://arxiv.org/abs/2302.12453 · HTML: https://mlsanity.com/paper/2302.12453/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 24 Feb 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- [Long-tailed Recognition by Routing Diverse Distribution-Aware Experts](https://mlsanity.com/paper/2010.01809/) (2020)
- [A Geometric Analysis of Neural Collapse with Unconstrained Features](https://mlsanity.com/paper/2105.02375/) (2021)
- [Parametric Contrastive Learning](https://mlsanity.com/paper/2107.12028/) (2021)
- [Improving Calibration for Long-Tailed Recognition](https://mlsanity.com/paper/2104.00466/) (2021)
- [Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training](https://mlsanity.com/paper/2101.12699/) (2021)
- [Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path](https://mlsanity.com/paper/2106.02073/) (2021)
- [Balanced Contrastive Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2207.09052/) (2022)
- [Targeted Supervised Contrastive Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2111.13998/) (2021)
- [Long-Tailed Recognition via Weight Balancing](https://mlsanity.com/paper/2203.14197/) (2022)
- [On the Role of Neural Collapse in Transfer Learning](https://mlsanity.com/paper/2112.15121/) (2021)
- [Neural Collapse: A Review on Modelling Principles and Generalization](https://mlsanity.com/paper/2206.04041/) (2022)
- [The Devil is in the Tails: Fine-grained Classification in the Wild](https://mlsanity.com/paper/1709.01450/) (2017)
- [Imbalance Trouble: Revisiting Neural-Collapse Geometry](https://mlsanity.com/paper/2208.05512/) (2022)
- [Deep Over-sampling Framework for Classifying Imbalanced Data](https://mlsanity.com/paper/1704.07515/) (2017)
- [Generalized Parametric Contrastive Learning](https://mlsanity.com/paper/2209.12400/) (2022)
- [Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learning](https://mlsanity.com/paper/2204.08735/) (2022)

## Built on by (most cited first, 28 of 28)

- [Generalized Neural Collapse for a Large Number of Classes](https://mlsanity.com/paper/2310.05351/) (2023)
- [Linguistic Collapse: Neural Collapse in (Large) Language Models](https://mlsanity.com/paper/2405.17767/) (2024)
- [Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Model](https://mlsanity.com/paper/2401.02058/) (2024)
- [Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity](https://mlsanity.com/paper/2303.05689/) (2023)
- [What Makes CLIP More Robust to Long-Tailed Pre-Training Data? A Controlled Study for Transferable Insights](https://mlsanity.com/paper/2405.21070/) (2024)
- [Progressive Feedforward Collapse of ResNet Training](https://mlsanity.com/paper/2405.00985/) (2024)
- [Cross Entropy versus Label Smoothing: A Neural Collapse Perspective](https://mlsanity.com/paper/2402.03979/) (2024)
- [Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers](https://mlsanity.com/paper/2505.15239/) (2025)
- [Control-oriented Clustering of Visual Latent Representation](https://mlsanity.com/paper/2410.05063/) (2024)
- [Exploring Weight Balancing on Long-Tailed Recognition Problem](https://mlsanity.com/paper/2305.16573/) (2023)
- [Towards Demystifying the Generalization Behaviors When Neural Collapse Emerges](https://mlsanity.com/paper/2310.08358/) (2023)
- [Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching](https://mlsanity.com/paper/2306.07960/) (2023)
- [Fractal Calibration for long-tailed object detection](https://mlsanity.com/paper/2410.11774/) (2024)
- [The Fair Language Model Paradox](https://mlsanity.com/paper/2410.11985/) (2024)
- [The Impact of Geometric Complexity on Neural Collapse in Transfer Learning](https://mlsanity.com/paper/2405.15706/) (2024)
- [MOKD: Cross-domain Finetuning for Few-shot Classification via Maximizing Optimized Kernel Dependence](https://mlsanity.com/paper/2405.18786/) (2024)
- [All-around Neural Collapse for Imbalanced Classification](https://mlsanity.com/paper/2408.07253/) (2024)
- [Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective](https://mlsanity.com/paper/2410.23391/) (2024)
- [Prior2Posterior: Model Prior Correction for Long-Tailed Learning](https://mlsanity.com/paper/2412.16540/) (2024)
- [Neural Collapse-Inspired Multi-Label Federated Learning under Label-Distribution Skew](https://mlsanity.com/paper/2509.12544/) (2025)
- [Space Alignment Matters: The Missing Piece for Inducing Neural Collapse in Long-Tailed Learning](https://mlsanity.com/paper/2512.07844/) (2025)
- [Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment](https://mlsanity.com/paper/2409.17582/) (2024)
- [Aligned Contrastive Loss for Long-Tailed Recognition](https://mlsanity.com/paper/2506.01071/) (2025)
- [Optimizer choice matters for the emergence of Neural Collapse](https://mlsanity.com/paper/2602.16642/) (2026)
- [rETF-semiSL: Semi-Supervised Learning for Neural Collapse in Temporal Data](https://mlsanity.com/paper/2508.10147/) (2025)
- [Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient](https://mlsanity.com/paper/2509.02737/) (2025)
- [Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of View](https://mlsanity.com/paper/2605.10047/) (2026)
- [The role of class encoding in neural collapse](https://mlsanity.com/paper/2606.00344/) (2026)

## Abstract (arXiv)

Although deep neural networks achieve tremendous success on various classification tasks, the generalization ability drops sheer when training datasets exhibit long-tailed distributions. One of the reasons is that the learned representations (i.e. features) from the imbalanced datasets are less effective than those from balanced datasets. Specifically, the learned representation under class-balanced distribution will present the Neural Collapse (NC) phenomena. NC indicates the features from the same category are close to each other and from different categories are maximally distant, showing an optimal linear separable state of classification. However, the pattern differs on imbalanced datasets and is partially responsible for the reduced performance of the model. In this work, we propose two explicit feature regularization terms to learn high-quality representation for class-imbalanced data. With the proposed regularization, NC phenomena will appear under the class-imbalanced distribution, and the generalization ability can be significantly improved. Our method is easily implemented, highly effective, and can be plugged into most existing methods. The extensive experimental results on widely-used benchmarks show the effectiveness of our method