Space Alignment Matters: The Missing Piece for Inducing Neural Collapse in Long-Tailed Learning
Not on the map. Why?
Abstract (arXiv)
Recent studies on Neural Collapse (NC) reveal that, under class-balanced conditions, the class feature means and classifier weights spontaneously align into a simplex equiangular tight frame (ETF). In long-tailed regimes, however, severe sample imbalance tends to prevent the emergence of the NC phenomenon, resulting in poor generalization performance. Current efforts predominantly seek to recover the ETF geometry by imposing constraints on features or classifier weights, yet overlook a critical problem: There is a pronounced misalignment between the feature and the classifier weight spaces. In this paper, we theoretically quantify the harm of such misalignment through an optimal error exponent analysis. Built on this insight, we propose three explicit alignment strategies that plug-and-play into existing long-tail methods without architectural change. Extensive experiments on the CIFAR-10-LT, CIFAR-100-LT, and ImageNet-LT datasets consistently boost examined baselines and achieve the state-of-the-art performances.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 25 Nov 2025.
Builds on
18 of 25 references.
References · 18
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
- 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
- 2020Long-tailed Recognition by Routing Diverse Distribution-Aware Expertscited by 271
- 2015Cost Sensitive Learning of Deep Feature Representations from Imbalanced Datacited by 167
- 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
- 2022Balanced Contrastive Learning for Long-Tailed Visual Recognitioncited by 144
- 2020Feature Space Augmentation for Long-Tailed Datacited by 136
- 2021Contrastive Learning based Hybrid Networks for Long-Tailed Image Classificationcited by 130
6 more
- 2015Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networkscited by 115
- 2022Long-Tailed Recognition via Weight Balancingcited by 107
- 2022Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learningcited by 43
- 2023Global and Local Mixture Consistency Cumulative Learning for Long-tailed Visual Recognitionscited by 43
- 2023Inducing Neural Collapse in Deep Long-tailed Learningcited by 28
- 2024Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Modelcited by 19
Built on by
Citing papers per year: 2026: 2.
2 papers.