# Constructing Taxonomies from Pretrained Language Models

arXiv: https://arxiv.org/abs/2010.12813 · HTML: https://mlsanity.com/paper/2010.12813/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2020.

## Built on by (most cited first, 21 of 21)

- [Organize the Web: Constructing Domains Enhances Pre-Training Data Curation](https://mlsanity.com/paper/2502.10341/) (2025)
- [A Vision Check-up for Language Models](https://mlsanity.com/paper/2401.01862/) (2024)
- [Indoor and Outdoor 3D Scene Graph Generation via Language-Enabled Spatial Ontologies](https://mlsanity.com/paper/2312.11713/) (2023)
- [Chain-of-Layer: Iteratively Prompting Large Language Models for Taxonomy Induction from Limited Examples](https://mlsanity.com/paper/2402.07386/) (2024)
- [Prompting or Fine-tuning? A Comparative Study of Large Language Models for Taxonomy Construction](https://mlsanity.com/paper/2309.01715/) (2023)
- [COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models](https://mlsanity.com/paper/2210.01963/) (2022)
- [Open Relation and Event Type Discovery with Type Abstraction](https://mlsanity.com/paper/2212.00178/) (2022)
- [Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field](https://mlsanity.com/paper/2412.08258/) (2024)
- [Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction](https://mlsanity.com/paper/2202.04876/) (2022)
- [From Papers to Panoramas: Building Hierarchies of Scientific Literature at Scale](https://mlsanity.com/paper/2504.13834/) (2025)
- [Hierarchical Selective Classification](https://mlsanity.com/paper/2405.11533/) (2024)
- [generAItor: Tree-in-the-Loop Text Generation for Language Model Explainability and Adaptation](https://mlsanity.com/paper/2403.07627/) (2024)
- [LLMTaxo: Leveraging Large Language Models for Constructing Taxonomy of Factual Claims from Social Media](https://mlsanity.com/paper/2504.12325/) (2025)
- [Ontology Completion with Natural Language Inference and Concept Embeddings: An Analysis](https://mlsanity.com/paper/2403.17216/) (2024)
- [GitRanking: A Ranking of GitHub Topics for Software Classification using Active Sampling](https://mlsanity.com/paper/2205.09379/) (2022)
- [Probing Pretrained Language Models with Hierarchy Properties](https://mlsanity.com/paper/2312.09670/) (2023)
- [Leveraging Large Language Models for Generating Research Topic Ontologies: A Multi-Disciplinary Study](https://mlsanity.com/paper/2508.20693/) (2025)
- [Hierarchical Text Classification with LLM-Refined Taxonomies](https://mlsanity.com/paper/2601.18375/) (2026)
- [CHisAgent: A Multi-Agent Framework for Event Taxonomy Construction in Ancient Chinese Cultural Systems](https://mlsanity.com/paper/2601.05520/) (2026)
- [A Hybrid AI Methodology for Generating Ontologies of Research Topics from Scientific Paper Corpora](https://mlsanity.com/paper/2508.04213/) (2025)
- [Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field](https://mlsanity.com/paper/2607.17902/) (2026)

## Abstract (arXiv)

We present a method for constructing taxonomic trees (e.g., WordNet) using pretrained language models. Our approach is composed of two modules, one that predicts parenthood relations and another that reconciles those predictions into trees. The parenthood prediction module produces likelihood scores for each potential parent-child pair, creating a graph of parent-child relation scores. The tree reconciliation module treats the task as a graph optimization problem and outputs the maximum spanning tree of this graph. We train our model on subtrees sampled from WordNet, and test on non-overlapping WordNet subtrees. We show that incorporating web-retrieved glosses can further improve performance. On the task of constructing subtrees of English WordNet, the model achieves 66.7 ancestor F1, a 20.0% relative increase over the previous best published result on this task. In addition, we convert the original English dataset into nine other languages using Open Multilingual WordNet and extend our results across these languages.