# Several Experiments on Investigating Pretraining and Knowledge-Enhanced Models for Natural Language Inference

arXiv: https://arxiv.org/abs/1904.12104 · HTML: https://mlsanity.com/paper/1904.12104/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2019.

## Built on by (most cited first, 6 of 6)

- [GERE: Generative Evidence Retrieval for Fact Verification](https://mlsanity.com/paper/2204.05511/) (2022)
- [Does External Knowledge Help Explainable Natural Language Inference? Automatic Evaluation vs. Human Ratings](https://mlsanity.com/paper/2109.07833/) (2021)
- [Natural Language Inference with Self-Attention for Veracity Assessment of Pandemic Claims](https://mlsanity.com/paper/2205.02596/) (2022)
- [How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding](https://mlsanity.com/paper/2109.05696/) (2021)
- [Unsupervised Pre-training with Structured Knowledge for Improving Natural Language Inference](https://mlsanity.com/paper/2109.03941/) (2021)
- [Multi-Evidence based Fact Verification via A Confidential Graph Neural Network](https://mlsanity.com/paper/2405.10481/) (2024)

## Abstract (arXiv)

Natural language inference (NLI) is among the most challenging tasks in natural language understanding. Recent work on unsupervised pretraining that leverages unsupervised signals such as language-model and sentence prediction objectives has shown to be very effective on a wide range of NLP problems. It would still be desirable to further understand how it helps NLI; e.g., if it learns artifacts in data annotation or instead learn true inference knowledge. In addition, external knowledge that does not exist in the limited amount of NLI training data may be added to NLI models in two typical ways, e.g., from human-created resources or an unsupervised pretraining paradigm. We runs several experiments here to investigate whether they help NLI in the same way, and if not,how?