# Learning Hierarchy Aware Features for Reducing Mistake Severity

arXiv: https://arxiv.org/abs/2207.12646 · HTML: https://mlsanity.com/paper/2207.12646/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 26 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks](https://mlsanity.com/paper/1912.09393/) (2019)
- [Do Convolutional Neural Networks Learn Class Hierarchy?](https://mlsanity.com/paper/1710.06501/) (2017)
- [Hierarchy-based Image Embeddings for Semantic Image Retrieval](https://mlsanity.com/paper/1809.09924/) (2018)
- [Hierarchical Text Classification with Reinforced Label Assignment](https://mlsanity.com/paper/1908.10419/) (2019)
- [Integrating domain knowledge: using hierarchies to improve deep classifiers](https://mlsanity.com/paper/1811.07125/) (2018)
- [No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks](https://mlsanity.com/paper/2104.00795/) (2021)
- [Leveraging Class Hierarchies with Metric-Guided Prototype Learning](https://mlsanity.com/paper/2007.03047/) (2020)
- [Hierarchical Proxy-based Loss for Deep Metric Learning](https://mlsanity.com/paper/2103.13538/) (2021)
- [Semi-Supervised Learning with Taxonomic Labels](https://mlsanity.com/paper/2111.11595/) (2021)
- [HIERMATCH: Leveraging Label Hierarchies for Improving Semi-Supervised Learning](https://mlsanity.com/paper/2111.00164/) (2021)

## Built on by (most cited first, 26 of 26)

- [Visually Consistent Hierarchical Image Classification](https://mlsanity.com/paper/2406.11608/) (2024)
- [Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity](https://mlsanity.com/paper/2303.05689/) (2023)
- [HELM: Hierarchical Encoding for mRNA Language Modeling](https://mlsanity.com/paper/2410.12459/) (2024)
- [Hyperbolic Random Forests](https://mlsanity.com/paper/2308.13279/) (2023)
- [Deep Imbalanced Regression via Hierarchical Classification Adjustment](https://mlsanity.com/paper/2310.17154/) (2023)
- [SEAL: Simultaneous Label Hierarchy Exploration And Learning](https://mlsanity.com/paper/2304.13374/) (2023)
- [Test-Time Amendment with a Coarse Classifier for Fine-Grained Classification](https://mlsanity.com/paper/2302.00368/) (2023)
- [Continual Hyperbolic Learning of Instances and Classes](https://mlsanity.com/paper/2506.10710/) (2025)
- [Out-of-Distribution Detection with Prototypical Outlier Proxy](https://mlsanity.com/paper/2412.16884/) (2024)
- [ProHOC: Probabilistic Hierarchical Out-of-Distribution Classification via Multi-Depth Networks](https://mlsanity.com/paper/2503.21397/) (2025)
- [Learning Structured Representations by Embedding Class Hierarchy with Fast Optimal Transport](https://mlsanity.com/paper/2410.03052/) (2024)
- [Free-Grained Hierarchical Visual Recognition](https://mlsanity.com/paper/2510.14737/) (2025)
- [Making Better Mistakes in CLIP-Based Zero-Shot Classification with Hierarchy-Aware Language Prompts](https://mlsanity.com/paper/2503.02248/) (2025)
- [BeetleVerse: A Study on Taxonomic Classification of Ground Beetles](https://mlsanity.com/paper/2504.13393/) (2025)
- [Beyond One-Hot-Encoding: Injecting Semantics to Drive Image Classifiers](https://mlsanity.com/paper/2308.00607/) (2023)
- [Harnessing Superclasses for Learning from Hierarchical Databases](https://mlsanity.com/paper/2411.16438/) (2024)
- [Priority-Aware Clinical Pathology Hierarchy Training for Multiple Instance Learning](https://mlsanity.com/paper/2507.20469/) (2025)
- [MATANet: A Multi-context Attention and Taxonomy-Aware Network for Fine-Grained Underwater Recognition of Marine Species](https://mlsanity.com/paper/2601.03729/) (2026)
- [Semi-Supervised Hierarchical Open-Set Classification](https://mlsanity.com/paper/2601.16541/) (2026)
- [Hier-COS: Making Deep Features Hierarchy-aware via Composition of Orthogonal Subspaces](https://mlsanity.com/paper/2503.07853/) (2025)
- [A Hierarchical Error Framework for Reliable Automated Coding in Communication Research: Applications to Health and Political Communication](https://mlsanity.com/paper/2509.24841/) (2025)
- [Semantics-Aware Hierarchical Consensus Learning for Remote Sensing Image Classification](https://mlsanity.com/paper/2510.04916/) (2025)
- [Every Error has Its Magnitude: Asymmetric Mistake Severity Training for Multiclass Multiple Instance Learning](https://mlsanity.com/paper/2603.13682/) (2026)
- [Online Continual Learning with Dynamic Label Hierarchies](https://mlsanity.com/paper/2605.11742/) (2026)
- [Hierarchical Scaffolding Enables Human-Like Cognitive Selectivity under Data Scarcity](https://mlsanity.com/paper/2607.04709/) (2026)
- [Subtype Robustness Is Not Just Accuracy: Calibration Under Unseen Subtype Shift](https://mlsanity.com/paper/2608.00928/) (2026)

## Abstract (arXiv)

Label hierarchies are often available apriori as part of biological taxonomy or language datasets WordNet. Several works exploit these to learn hierarchy aware features in order to improve the classifier to make semantically meaningful mistakes while maintaining or reducing the overall error. In this paper, we propose a novel approach for learning Hierarchy Aware Features (HAF) that leverages classifiers at each level of the hierarchy that are constrained to generate predictions consistent with the label hierarchy. The classifiers are trained by minimizing a Jensen-Shannon Divergence with target soft labels obtained from the fine-grained classifiers. Additionally, we employ a simple geometric loss that constrains the feature space geometry to capture the semantic structure of the label space. HAF is a training time approach that improves the mistakes while maintaining top-1 error, thereby, addressing the problem of cross-entropy loss that treats all mistakes as equal. We evaluate HAF on three hierarchical datasets and achieve state-of-the-art results on the iNaturalist-19 and CIFAR-100 datasets. The source code is available at https://github.com/07Agarg/HAF