# Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction

arXiv: https://arxiv.org/abs/2202.04876 · HTML: https://mlsanity.com/paper/2202.04876/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 10 Feb 2022.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model](https://mlsanity.com/paper/1902.04094/) (2019)
- [Unsupervised Commonsense Question Answering with Self-Talk](https://mlsanity.com/paper/2004.05483/) (2020)
- [Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2102.00894/) (2021)
- [Inducing Relational Knowledge from BERT](https://mlsanity.com/paper/1911.12753/) (2019)
- [BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?](https://mlsanity.com/paper/2105.04949/) (2021)
- [Prompting Contrastive Explanations for Commonsense Reasoning Tasks](https://mlsanity.com/paper/2106.06823/) (2021)
- [STEAM: Self-Supervised Taxonomy Expansion with Mini-Paths](https://mlsanity.com/paper/2006.10217/) (2020)
- [Self-supervised Knowledge Triplet Learning for Zero-shot Question Answering](https://mlsanity.com/paper/2005.00316/) (2020)
- [Constructing Taxonomies from Pretrained Language Models](https://mlsanity.com/paper/2010.12813/) (2020)
- [End-to-End Reinforcement Learning for Automatic Taxonomy Induction](https://mlsanity.com/paper/1805.04044/) (2018)
- [Contextualized Representations Using Textual Encyclopedic Knowledge](https://mlsanity.com/paper/2004.12006/) (2020)
- [Ask2Transformers: Zero-Shot Domain labelling with Pre-trained Language Models](https://mlsanity.com/paper/2101.02661/) (2021)
- [INRIASAC: Simple Hypernym Extraction Methods](https://mlsanity.com/paper/1502.01271/) (2015)
- [Mining Knowledge for Natural Language Inference from Wikipedia Categories](https://mlsanity.com/paper/2010.01239/) (2020)

## Built on by (most cited first, 8 of 8)

- [Chain-of-Layer: Iteratively Prompting Large Language Models for Taxonomy Induction from Limited Examples](https://mlsanity.com/paper/2402.07386/) (2024)
- [Prompting or Fine-tuning? A Comparative Study of Large Language Models for Taxonomy Construction](https://mlsanity.com/paper/2309.01715/) (2023)
- [TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Tasks](https://mlsanity.com/paper/2403.09207/) (2024)
- [From Papers to Panoramas: Building Hierarchies of Scientific Literature at Scale](https://mlsanity.com/paper/2504.13834/) (2025)
- [Probing Pretrained Language Models with Hierarchy Properties](https://mlsanity.com/paper/2312.09670/) (2023)
- [Exploring Prompt-Based Methods for Zero-Shot Hypernym Prediction with Large Language Models](https://mlsanity.com/paper/2401.04515/) (2024)
- [BoostTaxo: Zero-Shot Taxonomy Induction via Boosting-Style Agentic Reasoning and Constraint-Aware Calibration](https://mlsanity.com/paper/2605.12520/) (2026)
- [SPARROW: Scalable Taxonomy Induction via Structure-Preserving Partitioning and Constraint-Guided Merging](https://mlsanity.com/paper/2609.07307/) (2026)

## Abstract (arXiv)

In this paper, we analyze zero-shot taxonomy learning methods which are based on distilling knowledge from language models via prompting and sentence scoring. We show that, despite their simplicity, these methods outperform some supervised strategies and are competitive with the current state-of-the-art under adequate conditions. We also show that statistical and linguistic properties of prompts dictate downstream performance.