# Reference-Free Evaluation of Taxonomies

arXiv: https://arxiv.org/abs/2505.11470 · HTML: https://mlsanity.com/paper/2505.11470/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 46 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 18 of 18)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [GRAM: Graph-based Attention Model for Healthcare Representation Learning](https://mlsanity.com/paper/1611.07012/) (2016)
- [HDLTex: Hierarchical Deep Learning for Text Classification](https://mlsanity.com/paper/1709.08267/) (2017)
- [EL Embeddings: Geometric construction of models for the Description Logic EL ++](https://mlsanity.com/paper/1902.10499/) (2019)
- [OntoZSL: Ontology-enhanced Zero-shot Learning](https://mlsanity.com/paper/2102.07339/) (2021)
- [Taxonomy Completion via Triplet Matching Network](https://mlsanity.com/paper/2101.01896/) (2021)
- [Enhancing Taxonomy Completion with Concept Generation via Fusing Relational Representations](https://mlsanity.com/paper/2106.02974/) (2021)
- [Semantic Sensitivities and Inconsistent Predictions: Measuring the Fragility of NLI Models](https://mlsanity.com/paper/2401.14440/) (2024)
- [CodeTaxo: Enhancing Taxonomy Expansion with Limited Examples via Code Language Prompts](https://mlsanity.com/paper/2408.09070/) (2024)
- [Ontology-guided Semantic Composition for Zero-Shot Learning](https://mlsanity.com/paper/2006.16917/) (2020)
- [A compendium and evaluation of taxonomy quality attributes](https://mlsanity.com/paper/2403.00111/) (2024)
- [RaTE: a Reproducible automatic Taxonomy Evaluation by Filling the Gap](https://mlsanity.com/paper/2307.09706/) (2023)

## Abstract (arXiv)

We introduce two reference-free metrics for quality evaluation of taxonomies in the absence of labels. The first metric evaluates robustness by calculating the correlation between semantic and taxonomic similarity, addressing error types not considered by existing metrics. The second uses Natural Language Inference to assess logical adequacy. Both metrics are tested on five taxonomies and are shown to correlate well with F1 against ground truth taxonomies. We further demonstrate that our metrics can predict downstream performance in hierarchical classification when used with label hierarchies.