# Mining Knowledge for Natural Language Inference from Wikipedia Categories

arXiv: https://arxiv.org/abs/2010.01239 · HTML: https://mlsanity.com/paper/2010.01239/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2020.

## Built on by (most cited first, 2 of 2)

- [Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction](https://mlsanity.com/paper/2202.04876/) (2022)
- [Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answering](https://mlsanity.com/paper/2505.16591/) (2025)

## Abstract (arXiv)

Accurate lexical entailment (LE) and natural language inference (NLI) often require large quantities of costly annotations. To alleviate the need for labeled data, we introduce WikiNLI: a resource for improving model performance on NLI and LE tasks. It contains 428,899 pairs of phrases constructed from naturally annotated category hierarchies in Wikipedia. We show that we can improve strong baselines such as BERT and RoBERTa by pretraining them on WikiNLI and transferring the models on downstream tasks. We conduct systematic comparisons with phrases extracted from other knowledge bases such as WordNet and Wikidata to find that pretraining on WikiNLI gives the best performance. In addition, we construct WikiNLI in other languages, and show that pretraining on them improves performance on NLI tasks of corresponding languages.