# Pitfalls of Assessing Extracted Hierarchies for Multi-Class Classification

arXiv: https://arxiv.org/abs/2101.11095 · HTML: https://mlsanity.com/paper/2101.11095/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 37 references that are in the map.
- First posted on arXiv 26 Jan 2021.

## Builds on (most cited first, 2 of 2)

- [Bonsai -- Diverse and Shallow Trees for Extreme Multi-label Classification](https://mlsanity.com/paper/1904.08249/) (2019)
- [Joint Hierarchical Category Structure Learning and Large-Scale Image Classification](https://mlsanity.com/paper/1709.05072/) (2017)

## Built on by (most cited first, 2 of 2)

- [Constructing Multiclass Classifiers using Binary Classifiers Under Log-Loss](https://mlsanity.com/paper/2102.08184/) (2021)
- [Performance Improvement in Multi-class Classification via Automated Hierarchy Generation and Exploitation through Extended LCPN Schemes](https://mlsanity.com/paper/2310.20641/) (2023)

## Abstract (arXiv)

Using hierarchies of classes is one of the standard methods to solve multi-class classification problems. In the literature, selecting the right hierarchy is considered to play a key role in improving classification performance. Although different methods have been proposed, there is still a lack of understanding of what makes one method to extract hierarchies perform better or worse. To this effect, we analyze and compare some of the most popular approaches to extracting hierarchies. We identify some common pitfalls that may lead practitioners to make misleading conclusions about their methods. In addition, to address some of these problems, we demonstrate that using random hierarchies is an appropriate benchmark to assess how the hierarchy's quality affects the classification performance. In particular, we show how the hierarchy's quality can become irrelevant depending on the experimental setup: when using powerful enough classifiers, the final performance is not affected by the quality of the hierarchy. We also show how comparing the effect of the hierarchies against non-hierarchical approaches might incorrectly indicate their superiority. Our results confirm that datasets with a high number of classes generally present complex structures in how these classes relate to each other. In these datasets, the right hierarchy can dramatically improve classification performance.