mlsanityData to 7 Oct 2026

Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity

arXiv 2303.05689 ↗First posted 10 Mar 2023cs.CV cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

There is a recently discovered and intriguing phenomenon called Neural Collapse: at the terminal phase of training a deep neural network for classification, the within-class penultimate feature means and the associated classifier vectors of all flat classes collapse to the vertices of a simplex Equiangular Tight Frame (ETF). Recent work has tried to exploit this phenomenon by fixing the related classifier weights to a pre-computed ETF to induce neural collapse and maximize the separation of the learned features when training with imbalanced data. In this work, we propose to fix the linear classifier of a deep neural network to a Hierarchy-Aware Frame (HAFrame), instead of an ETF, and use a cosine similarity-based auxiliary loss to learn hierarchy-aware penultimate features that collapse to the HAFrame. We demonstrate that our approach reduces the mistake severity of the model's predictions while maintaining its top-1 accuracy on several datasets of varying scales with hierarchies of heights ranging from 3 to 12. Code: https://github.com/ltong1130ztr/HAFrame

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 32 of its 48 references that are in the map. First posted on arXiv 10 Mar 2023.

Builds on

32 of 48 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  7. 2016Wide Residual Networkscited by 3,057
  8. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  9. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  10. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  11. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  12. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
20 more
  1. 2013Zero-Shot Learning by Convex Combination of Semantic Embeddingscited by 135
  2. 2019Making Better Mistakes: Leveraging Class Hierarchies with Deep Networkscited by 112
  3. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  4. 2017Do Convolutional Neural Networks Learn Class Hierarchy?cited by 61
  5. 2021PASS: An ImageNet replacement for self-supervised pretraining without humanscited by 51
  6. 2023Understanding Imbalanced Semantic Segmentation Through Neural Collapsecited by 46
  7. 2018Fix your classifier: the marginal value of training the last weight layercited by 43
  8. 2018Hierarchy-based Image Embeddings for Semantic Image Retrievalcited by 41
  9. 2018Dynamic-structured Semantic Propagation Networkcited by 38
  10. 2020Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifiercited by 34
  11. 2023Inducing Neural Collapse in Deep Long-tailed Learningcited by 28
  12. 2018Hierarchical Novelty Detection for Visual Object Recognitioncited by 26
  13. 2022Learning Hierarchy Aware Features for Reducing Mistake Severitycited by 26
  14. 2018Integrating domain knowledge: using hierarchies to improve deep classifierscited by 25
  15. 2021Regular Polytope Networkscited by 24
  16. 2021No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networkscited by 21
  17. 2022Skin Lesion Recognition with Class-Hierarchy Regularized Hyperbolic Embeddingscited by 19
  18. 2023Maximally Compact and Separated Features with Regular Polytope Networkscited by 5
  19. 2021Embedding Semantic Hierarchy in Discrete Optimal Transport for Risk Minimizationcited by 4
  20. 2021Connecting Sphere Manifolds Hierarchically for Regularizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 5; 2026: 3.

13 papers.

Sort:

  1. 2023Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Modelcited by 43
  2. 2024Linguistic Collapse: Neural Collapse in (Large) Language Modelscited by 36
  3. 2025Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformerscited by 9
  4. 2023Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batchingcited by 5
  5. 2024Out-of-Distribution Detection with Prototypical Outlier Proxycited by 4
  6. 2025ProHOC: Probabilistic Hierarchical Out-of-Distribution Classification via Multi-Depth Networkscited by 4
  7. 2025Making Better Mistakes in CLIP-Based Zero-Shot Classification with Hierarchy-Aware Language Promptscited by 2
  8. 2023It's an Alignment, Not a Trade-off: Revisiting Bias and Variance in Deep Modelscited by 1
  9. 2026Semi-Supervised Hierarchical Open-Set Classificationcited by 1
  10. 2025Hier-COS: Making Deep Features Hierarchy-aware via Composition of Orthogonal Subspacescited by 0
  11. 2025Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradientcited by 0
  12. 2026Online Continual Learning with Dynamic Label Hierarchiescited by 0
1 more
  1. 2026RST: certifying and constructing prescribed information in variational autoencoderscited by 0