# Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks

arXiv: https://arxiv.org/abs/1912.09393 · HTML: https://mlsanity.com/paper/1912.09393/

## Key facts

- Cited by 113 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2019.

## Built on by (most cited first, 30 of 113)

- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [Deep Hierarchical Semantic Segmentation](https://mlsanity.com/paper/2203.14335/) (2022)
- [Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework](https://mlsanity.com/paper/2204.13207/) (2022)
- [Benchmarking Omni-Vision Representation through the Lens of Visual Realms](https://mlsanity.com/paper/2207.07106/) (2022)
- [Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing](https://mlsanity.com/paper/2210.15889/) (2022)
- [Learning Hierarchy Aware Features for Reducing Mistake Severity](https://mlsanity.com/paper/2207.12646/) (2022)
- [No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks](https://mlsanity.com/paper/2104.00795/) (2021)
- [Integrating Language Guidance into Vision-based Deep Metric Learning](https://mlsanity.com/paper/2203.08543/) (2022)
- [Skin Lesion Recognition with Class-Hierarchy Regularized Hyperbolic Embeddings](https://mlsanity.com/paper/2209.05842/) (2022)
- [Emergent Visual-Semantic Hierarchies in Image-Text Representations](https://mlsanity.com/paper/2407.08521/) (2024)
- [LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning](https://mlsanity.com/paper/2309.13556/) (2023)
- [Visually Consistent Hierarchical Image Classification](https://mlsanity.com/paper/2406.11608/) (2024)
- [Improving Dietary Assessment Via Integrated Hierarchy Food Classification](https://mlsanity.com/paper/2109.02736/) (2021)
- [Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition](https://mlsanity.com/paper/2107.07029/) (2021)
- [Hierarchical classification at multiple operating points](https://mlsanity.com/paper/2210.10929/) (2022)
- [Continual Learning with Evolving Class Ontologies](https://mlsanity.com/paper/2210.04993/) (2022)
- [Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity](https://mlsanity.com/paper/2303.05689/) (2023)
- [LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies](https://mlsanity.com/paper/2407.16067/) (2024)
- [Interpretable part-whole hierarchies and conceptual-semantic relationships in neural networks](https://mlsanity.com/paper/2203.03282/) (2022)
- [Online Continual Learning on Hierarchical Label Expansion](https://mlsanity.com/paper/2308.14374/) (2023)
- [SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detection](https://mlsanity.com/paper/2405.10053/) (2024)
- [Taxonomy-Aware Evaluation of Vision-Language Models](https://mlsanity.com/paper/2504.05457/) (2025)
- [Revisiting Single-gated Mixtures of Experts](https://mlsanity.com/paper/2304.05497/) (2023)
- [Optimization of Rank Losses for Image Retrieval](https://mlsanity.com/paper/2309.08250/) (2023)
- [HELM: Hierarchical Encoding for mRNA Language Modeling](https://mlsanity.com/paper/2410.12459/) (2024)
- [Deep Imbalanced Regression via Hierarchical Classification Adjustment](https://mlsanity.com/paper/2310.17154/) (2023)
- [Flattening the Parent Bias: Hierarchical Semantic Segmentation in the Poincar'e Ball](https://mlsanity.com/paper/2404.03778/) (2024)
- [Multi-Granularity Regularized Re-Balancing for Class Incremental Learning](https://mlsanity.com/paper/2206.15189/) (2022)
- [Hierarchical Average Precision Training for Pertinent Image Retrieval](https://mlsanity.com/paper/2207.04873/) (2022)
- [Hierarchical Few-Shot Object Detection: Problem, Benchmark and Method](https://mlsanity.com/paper/2210.03940/) (2022)

## Abstract (arXiv)

Deep neural networks have improved image classification dramatically over the past decade, but have done so by focusing on performance measures that treat all classes other than the ground truth as equally wrong. This has led to a situation in which mistakes are less likely to be made than before, but are equally likely to be absurd or catastrophic when they do occur. Past works have recognised and tried to address this issue of mistake severity, often by using graph distances in class hierarchies, but this has largely been neglected since the advent of the current deep learning era in computer vision. In this paper, we aim to renew interest in this problem by reviewing past approaches and proposing two simple modifications of the cross-entropy loss which outperform the prior art under several metrics on two large datasets with complex class hierarchies: tieredImageNet and iNaturalist'19.