# From the One, Judge of the Whole: Typed Entailment Graph Construction with Predicate Generation

arXiv: https://arxiv.org/abs/2306.04170 · HTML: https://mlsanity.com/paper/2306.04170/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data](https://mlsanity.com/paper/2009.09139/) (2020)
- [Probabilistic Models of Relational Implication](https://mlsanity.com/paper/1907.12048/) (2019)
- [Entailment Graph Learning with Textual Entailment and Soft Transitivity](https://mlsanity.com/paper/2204.03286/) (2022)
- [Multivalent Entailment Graphs for Question Answering](https://mlsanity.com/paper/2104.07846/) (2021)
- [SherLIiC: A Typed Event-Focused Lexical Inference Benchmark for Evaluating Natural Language Inference](https://mlsanity.com/paper/1906.01393/) (2019)
- [Smoothing Entailment Graphs with Language Models](https://mlsanity.com/paper/2208.00318/) (2022)
- [Incorporating Temporal Information in Entailment Graph Mining](https://mlsanity.com/paper/2109.09412/) (2021)
- [Combining Axiom Injection and Knowledge Base Completion for Efficient Natural Language Inference](https://mlsanity.com/paper/1811.06203/) (2018)
- [Large-scale Taxonomy Induction Using Entity and Word Embeddings](https://mlsanity.com/paper/2105.01305/) (2021)

## Abstract (arXiv)

Entailment Graphs (EGs) have been constructed based on extracted corpora as a strong and explainable form to indicate context-independent entailment relations in natural languages. However, EGs built by previous methods often suffer from the severe sparsity issues, due to limited corpora available and the long-tail phenomenon of predicate distributions. In this paper, we propose a multi-stage method, Typed Predicate-Entailment Graph Generator (TP-EGG), to tackle this problem. Given several seed predicates, TP-EGG builds the graphs by generating new predicates and detecting entailment relations among them. The generative nature of TP-EGG helps us leverage the recent advances from large pretrained language models (PLMs), while avoiding the reliance on carefully prepared corpora. Experiments on benchmark datasets show that TP-EGG can generate high-quality and scale-controllable entailment graphs, achieving significant in-domain improvement over state-of-the-art EGs and boosting the performance of down-stream inference tasks.