# A compendium and evaluation of taxonomy quality attributes

arXiv: https://arxiv.org/abs/2403.00111 · HTML: https://mlsanity.com/paper/2403.00111/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Feb 2024.

## Built on by (most cited first, 6 of 6)

- [Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applications](https://mlsanity.com/paper/2406.10300/) (2024)
- [LITE: LLM-Impelled efficient Taxonomy Evaluation](https://mlsanity.com/paper/2504.01369/) (2025)
- [Do Sparse Autoencoders Learn Meaningful Concept Hierarchies?](https://mlsanity.com/paper/2606.22994/) (2026)
- [Reference-Free Evaluation of Taxonomies](https://mlsanity.com/paper/2505.11470/) (2025)
- [A categorical error sensitivity index (ISEC): A preventive ordinal decision-support measure for irrecoverable errors in manual data entry systems](https://mlsanity.com/paper/2605.12328/) (2026)
- [WiCleanData: Guaranteeing the Type Consistency of Wikidata by Taxonomy Refinement and Constraint Enforcement](https://mlsanity.com/paper/2609.20057/) (2026)

## Abstract (arXiv)

Introduction: Taxonomies capture knowledge about a particular domain in a succinct manner and establish a common understanding among peers. Researchers use taxonomies to convey information about a particular knowledge area or to support automation tasks, and practitioners use them to enable communication beyond organizational boundaries. Aims: Despite this important role of taxonomies in software engineering, their quality is seldom evaluated. Our aim is to identify and define taxonomy quality attributes that provide practical measurements, helping researchers and practitioners to compare taxonomies and choose the one most adequate for the task at hand. Methods: We reviewed 324 publications from software engineering and information systems research and synthesized, when provided, the definitions of quality attributes and measurements. We evaluated the usefulness of the measurements on six taxonomies from three domains. Results: We propose the definition of seven quality attributes and suggest internal and external measurements that can be used to assess a taxonomy's quality. For two measurements we provide implementations in Python. We found the measurements useful for deciding which taxonomy is best suited for a particular purpose. Conclusion: While there exist several guidelines for creating taxonomies, there is a lack of actionable criteria to compare taxonomies. In this paper, we fill this gap by synthesizing from a wealth of literature seven, non-overlapping taxonomy quality attributes and corresponding measurements. Future work encompasses their further evaluation of usefulness and empirical validation.