# Evaluation Measures for Hierarchical Classification: a unified view and novel approaches

arXiv: https://arxiv.org/abs/1306.6802 · HTML: https://mlsanity.com/paper/1306.6802/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2013.

## Built on by (most cited first, 30 of 46)

- [Compositional Entailment Learning for Hyperbolic Vision-Language Models](https://mlsanity.com/paper/2410.06912/) (2024)
- [A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice](https://mlsanity.com/paper/2404.16958/) (2024)
- [Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering](https://mlsanity.com/paper/2106.14885/) (2021)
- [Hierarchical classification at multiple operating points](https://mlsanity.com/paper/2210.10929/) (2022)
- [Overview of BioASQ 2020: The eighth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering](https://mlsanity.com/paper/2106.14618/) (2021)
- [Can GPT-3.5 Generate and Code Discharge Summaries?](https://mlsanity.com/paper/2401.13512/) (2024)
- [Taxonomy-Aware Evaluation of Vision-Language Models](https://mlsanity.com/paper/2504.05457/) (2025)
- [CoPHE: A Count-Preserving Hierarchical Evaluation Metric in Large-Scale Multi-Label Text Classification](https://mlsanity.com/paper/2109.04853/) (2021)
- [A Bayesian Bradley-Terry model to compare multiple ML algorithms on multiple data sets](https://mlsanity.com/paper/2208.04935/) (2022)
- [Overview of BioASQ 2022: The tenth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering](https://mlsanity.com/paper/2210.06852/) (2022)
- [HunFlair2 in a cross-corpus evaluation of biomedical named entity recognition and normalization tools](https://mlsanity.com/paper/2402.12372/) (2024)
- [Can Current Explainability Help Provide References in Clinical Notes to Support Humans Annotate Medical Codes?](https://mlsanity.com/paper/2210.15882/) (2022)
- [Revisiting Hierarchical Text Classification: Inference and Metrics](https://mlsanity.com/paper/2410.01305/) (2024)
- [The LLM Bottleneck: Why Open-Source Vision LLMs Struggle with Hierarchical Visual Recognition](https://mlsanity.com/paper/2505.24840/) (2025)
- [PHyCLIP: ℓ1-Product of Hyperbolic Factors Unifies Hierarchy and Compositionality in Vision-Language Representation Learning](https://mlsanity.com/paper/2510.08919/) (2025)
- [Hierarchical confusion matrix for classification performance evaluation](https://mlsanity.com/paper/2306.09461/) (2023)
- [On the privacy-utility trade-off in differentially private hierarchical text classification](https://mlsanity.com/paper/2103.02895/) (2021)
- [Embedding Semantic Hierarchy in Discrete Optimal Transport for Risk Minimization](https://mlsanity.com/paper/2105.00101/) (2021)
- [Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models](https://mlsanity.com/paper/2603.00431/) (2026)
- [United We Learn Better: Harvesting Learning Improvements From Class Hierarchies Across Tasks](https://mlsanity.com/paper/2107.13627/) (2021)
- [Exploiting Expert Knowledge for Assigning Firms to Industries: A Novel Deep Learning Method](https://mlsanity.com/paper/2209.05943/) (2022)
- [Leveraging Taxonomy and LLMs for Improved Multimodal Hierarchical Classification](https://mlsanity.com/paper/2501.06827/) (2025)
- [Majority Rules: LLM Ensemble is a Winning Approach for Content Categorization](https://mlsanity.com/paper/2511.15714/) (2025)
- [Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer](https://mlsanity.com/paper/2606.11018/) (2026)
- [Hierarchical Scoring for Machine Learning Classifier Error Impact Evaluation](https://mlsanity.com/paper/2508.04489/) (2025)
- [Multilingual Clinical NER for Diseases and Medications Recognition in Cardiology Texts using BERT Embeddings](https://mlsanity.com/paper/2510.17437/) (2025)
- [MedPath: Multi-Domain Cross-Vocabulary Hierarchical Paths for Biomedical Entity Linking](https://mlsanity.com/paper/2511.10887/) (2025)
- [Semi-Supervised Hierarchical Open-Set Classification](https://mlsanity.com/paper/2601.16541/) (2026)
- [Evaluating Open-Source LLMs for Multi-Label ATT&CK Technique Classification on CTI Reports](https://mlsanity.com/paper/2606.18166/) (2026)
- [KVpop -- Key-Value Cache Compression with Predictive Online Pruning](https://mlsanity.com/paper/2607.05061/) (2026)

## Abstract (arXiv)

Hierarchical classification addresses the problem of classifying items into a hierarchy of classes. An important issue in hierarchical classification is the evaluation of different classification algorithms, which is complicated by the hierarchical relations among the classes. Several evaluation measures have been proposed for hierarchical classification using the hierarchy in different ways. This paper studies the problem of evaluation in hierarchical classification by analyzing and abstracting the key components of the existing performance measures. It also proposes two alternative generic views of hierarchical evaluation and introduces two corresponding novel measures. The proposed measures, along with the state-of-the art ones, are empirically tested on three large datasets from the domain of text classification. The empirical results illustrate the undesirable behavior of existing approaches and how the proposed methods overcome most of these methods across a range of cases.