# Test-Time Amendment with a Coarse Classifier for Fine-Grained Classification

arXiv: https://arxiv.org/abs/2302.00368 · HTML: https://mlsanity.com/paper/2302.00368/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 1 Feb 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Hyperbolic Image Embeddings](https://mlsanity.com/paper/1904.02239/) (2019)
- [Latent Embeddings for Zero-shot Classification](https://mlsanity.com/paper/1603.08895/) (2016)
- [Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks](https://mlsanity.com/paper/1912.09393/) (2019)
- [Part-based R-CNNs for Fine-grained Category Detection](https://mlsanity.com/paper/1407.3867/) (2014)
- [Do Convolutional Neural Networks Learn Class Hierarchy?](https://mlsanity.com/paper/1710.06501/) (2017)
- [Hierarchy-based Image Embeddings for Semantic Image Retrieval](https://mlsanity.com/paper/1809.09924/) (2018)
- [Wisdom of Committees: An Overlooked Approach To Faster and More Accurate Models](https://mlsanity.com/paper/2012.01988/) (2020)
- [A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification](https://mlsanity.com/paper/2104.00679/) (2021)
- [Learning Hierarchy Aware Features for Reducing Mistake Severity](https://mlsanity.com/paper/2207.12646/) (2022)
- [No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks](https://mlsanity.com/paper/2104.00795/) (2021)
- [Leveraging Class Hierarchies with Metric-Guided Prototype Learning](https://mlsanity.com/paper/2007.03047/) (2020)

## Built on by (most cited first, 5 of 5)

- [TransHP: Image Classification with Hierarchical Prompting](https://mlsanity.com/paper/2304.06385/) (2023)
- [Visually Consistent Hierarchical Image Classification](https://mlsanity.com/paper/2406.11608/) (2024)
- [ProHOC: Probabilistic Hierarchical Out-of-Distribution Classification via Multi-Depth Networks](https://mlsanity.com/paper/2503.21397/) (2025)
- [Making Better Mistakes in CLIP-Based Zero-Shot Classification with Hierarchy-Aware Language Prompts](https://mlsanity.com/paper/2503.02248/) (2025)
- [Semi-Supervised Hierarchical Open-Set Classification](https://mlsanity.com/paper/2601.16541/) (2026)

## Abstract (arXiv)

We investigate the problem of reducing mistake severity for fine-grained classification. Fine-grained classification can be challenging, mainly due to the requirement of domain expertise for accurate annotation. However, humans are particularly adept at performing coarse classification as it requires relatively low levels of expertise. To this end, we present a novel approach for Post-Hoc Correction called Hierarchical Ensembles (HiE) that utilizes label hierarchy to improve the performance of fine-grained classification at test-time using the coarse-grained predictions. By only requiring the parents of leaf nodes, our method significantly reduces avg. mistake severity while improving top-1 accuracy on the iNaturalist-19 and tieredImageNet-H datasets, achieving a new state-of-the-art on both benchmarks. We also investigate the efficacy of our approach in the semi-supervised setting. Our approach brings notable gains in top-1 accuracy while significantly decreasing the severity of mistakes as training data decreases for the fine-grained classes. The simplicity and post-hoc nature of HiE renders it practical to be used with any off-the-shelf trained model to improve its predictions further.