# Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity

arXiv: https://arxiv.org/abs/2303.05689 · HTML: https://mlsanity.com/paper/2303.05689/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 48 references that are in the map.
- First posted on arXiv 10 Mar 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training](https://mlsanity.com/paper/2101.12699/) (2021)
- [Zero-Shot Learning by Convex Combination of Semantic Embeddings](https://mlsanity.com/paper/1312.5650/) (2013)
- [Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks](https://mlsanity.com/paper/1912.09393/) (2019)
- [Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?](https://mlsanity.com/paper/2203.09081/) (2022)
- [Do Convolutional Neural Networks Learn Class Hierarchy?](https://mlsanity.com/paper/1710.06501/) (2017)
- [PASS: An ImageNet replacement for self-supervised pretraining without humans](https://mlsanity.com/paper/2109.13228/) (2021)
- [Understanding Imbalanced Semantic Segmentation Through Neural Collapse](https://mlsanity.com/paper/2301.01100/) (2023)
- [Fix your classifier: the marginal value of training the last weight layer](https://mlsanity.com/paper/1801.04540/) (2018)
- [Hierarchy-based Image Embeddings for Semantic Image Retrieval](https://mlsanity.com/paper/1809.09924/) (2018)
- [Dynamic-structured Semantic Propagation Network](https://mlsanity.com/paper/1803.06067/) (2018)
- [Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier](https://mlsanity.com/paper/2007.09898/) (2020)
- [Inducing Neural Collapse in Deep Long-tailed Learning](https://mlsanity.com/paper/2302.12453/) (2023)
- [Hierarchical Novelty Detection for Visual Object Recognition](https://mlsanity.com/paper/1804.00722/) (2018)
- [Learning Hierarchy Aware Features for Reducing Mistake Severity](https://mlsanity.com/paper/2207.12646/) (2022)
- [Integrating domain knowledge: using hierarchies to improve deep classifiers](https://mlsanity.com/paper/1811.07125/) (2018)
- [Regular Polytope Networks](https://mlsanity.com/paper/2103.15632/) (2021)
- [No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks](https://mlsanity.com/paper/2104.00795/) (2021)
- [Skin Lesion Recognition with Class-Hierarchy Regularized Hyperbolic Embeddings](https://mlsanity.com/paper/2209.05842/) (2022)
- [Maximally Compact and Separated Features with Regular Polytope Networks](https://mlsanity.com/paper/2301.06116/) (2023)

## Built on by (most cited first, 13 of 13)

- [Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model](https://mlsanity.com/paper/2305.13165/) (2023)
- [Linguistic Collapse: Neural Collapse in (Large) Language Models](https://mlsanity.com/paper/2405.17767/) (2024)
- [Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers](https://mlsanity.com/paper/2505.15239/) (2025)
- [Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching](https://mlsanity.com/paper/2306.07960/) (2023)
- [Out-of-Distribution Detection with Prototypical Outlier Proxy](https://mlsanity.com/paper/2412.16884/) (2024)
- [ProHOC: Probabilistic Hierarchical Out-of-Distribution Classification via Multi-Depth Networks](https://mlsanity.com/paper/2503.21397/) (2025)
- [Making Better Mistakes in CLIP-Based Zero-Shot Classification with Hierarchy-Aware Language Prompts](https://mlsanity.com/paper/2503.02248/) (2025)
- [It's an Alignment, Not a Trade-off: Revisiting Bias and Variance in Deep Models](https://mlsanity.com/paper/2310.09250/) (2023)
- [Semi-Supervised Hierarchical Open-Set Classification](https://mlsanity.com/paper/2601.16541/) (2026)
- [Hier-COS: Making Deep Features Hierarchy-aware via Composition of Orthogonal Subspaces](https://mlsanity.com/paper/2503.07853/) (2025)
- [Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient](https://mlsanity.com/paper/2509.02737/) (2025)
- [Online Continual Learning with Dynamic Label Hierarchies](https://mlsanity.com/paper/2605.11742/) (2026)
- [RST: certifying and constructing prescribed information in variational autoencoders](https://mlsanity.com/paper/2605.18224/) (2026)

## Abstract (arXiv)

There is a recently discovered and intriguing phenomenon called Neural Collapse: at the terminal phase of training a deep neural network for classification, the within-class penultimate feature means and the associated classifier vectors of all flat classes collapse to the vertices of a simplex Equiangular Tight Frame (ETF). Recent work has tried to exploit this phenomenon by fixing the related classifier weights to a pre-computed ETF to induce neural collapse and maximize the separation of the learned features when training with imbalanced data. In this work, we propose to fix the linear classifier of a deep neural network to a Hierarchy-Aware Frame (HAFrame), instead of an ETF, and use a cosine similarity-based auxiliary loss to learn hierarchy-aware penultimate features that collapse to the HAFrame. We demonstrate that our approach reduces the mistake severity of the model's predictions while maintaining its top-1 accuracy on several datasets of varying scales with hierarchies of heights ranging from 3 to 12. Code: https://github.com/ltong1130ztr/HAFrame