# Adversarial NLI for Factual Correctness in Text Summarisation Models

arXiv: https://arxiv.org/abs/2005.11739 · HTML: https://mlsanity.com/paper/2005.11739/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2020.

## Built on by (most cited first, 13 of 13)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization](https://mlsanity.com/paper/2112.08542/) (2021)
- [A Critical Evaluation of Evaluations for Long-form Question Answering](https://mlsanity.com/paper/2305.18201/) (2023)
- [Safeguarding Large Language Models: A Survey](https://mlsanity.com/paper/2406.02622/) (2024)
- [Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods](https://mlsanity.com/paper/2203.05227/) (2022)
- [Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search](https://mlsanity.com/paper/2203.08436/) (2022)
- [Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries](https://mlsanity.com/paper/2109.09195/) (2021)
- [Evaluating Large Language Models for Health-related Queries with Presuppositions](https://mlsanity.com/paper/2312.08800/) (2023)
- [SWING: Balancing Coverage and Faithfulness for Dialogue Summarization](https://mlsanity.com/paper/2301.10483/) (2023)
- [Using Similarity to Evaluate Factual Consistency in Summaries](https://mlsanity.com/paper/2409.15090/) (2024)
- [Just ClozE! A Novel Framework for Evaluating the Factual Consistency Faster in Abstractive Summarization](https://mlsanity.com/paper/2210.02804/) (2022)
- [Improving Factual Error Correction for Abstractive Summarization via Data Distillation and Conditional-generation Cloze](https://mlsanity.com/paper/2402.08581/) (2024)

## Abstract (arXiv)

We apply the Adversarial NLI dataset to train the NLI model and show that the model has the potential to enhance factual correctness in abstract summarization. We follow the work of Falke et al. (2019), which rank multiple generated summaries based on the entailment probabilities between an source document and summaries and select the summary that has the highest entailment probability. The authors' earlier study concluded that current NLI models are not sufficiently accurate for the ranking task. We show that the Transformer models fine-tuned on the new dataset achieve significantly higher accuracy and have the potential of selecting a coherent summary.