# For Generated Text, Is NLI-Neutral Text the Best Text?

arXiv: https://arxiv.org/abs/2302.08577 · HTML: https://mlsanity.com/paper/2302.08577/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 16 Feb 2023.

## Builds on (most cited first, 11 of 11)

- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Learning to Write with Cooperative Discriminators](https://mlsanity.com/paper/1805.06087/) (2018)
- [Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text](https://mlsanity.com/paper/2107.01294/) (2021)
- [Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning](https://mlsanity.com/paper/2107.02794/) (2021)
- [Generating Persona Consistent Dialogues by Exploiting Natural Language Inference](https://mlsanity.com/paper/1911.05889/) (2019)
- [Pragmatically Informative Text Generation](https://mlsanity.com/paper/1904.01301/) (2019)
- [Semantic Diversity in Dialogue with Natural Language Inference](https://mlsanity.com/paper/2205.01497/) (2022)
- [Entailment Semantics Can Be Extracted from an Ideal Language Model](https://mlsanity.com/paper/2209.12407/) (2022)

## Built on by (most cited first, 1 of 1)

- [Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains](https://mlsanity.com/paper/2410.08327/) (2024)

## Abstract (arXiv)

We explore incorporating natural language inference (NLI) into the text generative pipeline by using a pre-trained NLI model to assess whether a generated sentence entails, contradicts, or is neutral to the prompt and preceding text. First, we show that the NLI task is predictive of generation errors made by GPT-3. We use these results to develop an NLI-informed generation procedure for GPT-J. Then, we evaluate these generations by obtaining human annotations on error types and overall quality. We find that an NLI strategy of maximizing entailment improves text generation when the nucleus sampling randomness parameter value is high, while one which maximizes contradiction is in fact productive when the parameter value is low. Overall, though, we demonstrate that an NLI strategy of maximizing the neutral class provides the highest quality of generated text (significantly better than the vanilla generations), regardless of parameter value.