mlsanityData to 7 Oct 2026

Semantic Diversity in Dialogue with Natural Language Inference

arXiv 2205.01497 ↗First posted 3 May 2022cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

Generating diverse, interesting responses to chitchat conversations is a problem for neural conversational agents. This paper makes two substantial contributions to improving diversity in dialogue generation. First, we propose a novel metric which uses Natural Language Inference (NLI) to measure the semantic diversity of a set of model responses for a conversation. We evaluate this metric using an established framework (Tevet and Berant, 2021) and find strong evidence indicating NLI Diversity is correlated with semantic diversity. Specifically, we show that the contradiction relation is more useful than the neutral relation for measuring this diversity and that incorporating the NLI model's confidence achieves state-of-the-art results. Second, we demonstrate how to iteratively improve the semantic diversity of a sampled set of responses via a new generation procedure called Diversity Threshold Generation, which results in an average 137% increase in NLI Diversity compared to standard generation procedures.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 3 May 2022.

Builds on

26 of 37 references.

References · 26

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  9. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  10. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  11. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  12. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
14 more
  1. 2020Recipes for building an open-domain chatbotcited by 593
  2. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  3. 2019What makes a good conversation? How controllable attributes affect human judgmentscited by 140
  4. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
  5. 2020Evaluating the Evaluation of Diversity in Natural Language Generationcited by 127
  6. 2018Combining Fact Extraction and Verification with Neural Semantic Matching Networkscited by 120
  7. 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 114
  8. 2020Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretrainingcited by 59
  9. 2019Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple Referencescited by 48
  10. 2019Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systemscited by 42
  11. 2019Generating Persona Consistent Dialogues by Exploiting Natural Language Inferencecited by 33
  12. 2019Outlier Detection for Improved Data Quality and Diversity in Dialog Systemscited by 28
  13. 2019Syntax-Infused Variational Autoencoder for Text Generationcited by 19
  14. 2017Latent Variable Dialogue Models and their Diversitycited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 6; 2025: 7; 2026: 6.

28 papers.

Sort:

By area:

  1. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  2. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 97
  3. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  4. 2023What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variabilitycited by 51
  5. 2025Evaluating the Diversity and Quality of LLM Generated Contentcited by 51
  6. 2025Flipping the Dialogue: Training and Evaluating User Language Modelscited by 50
  7. 2024Benchmarking Linguistic Diversity of Large Language Modelscited by 37
  8. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  9. 2023How Far Can We Extract Diverse Perspectives from Large Language Models?cited by 22
  10. 2024Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioningcited by 18
  11. 2024Evaluating Diversity in Automatic Poetry Generationcited by 17
  12. 2022Measuring and Improving Semantic Diversity of Dialogue Generationcited by 16
16 more
  1. 2024Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Taskscited by 12
  2. 2026Where does output diversity collapse in post-training?cited by 10
  3. 2025Optimizing Diversity and Quality through Base-Aligned Model Collaborationcited by 8
  4. 2022Best-k Search Algorithm for Neural Text Generationcited by 6
  5. 2025DESIGNER: Design-Logic-Guided Multidisciplinary Data Synthesis for LLM Reasoningcited by 6
  6. 2025Measuring Diversity in Synthetic Datasetscited by 5
  7. 2026Limited Linguistic Diversity in Embodied AI Datasetscited by 5
  8. 2024Exploring Continual Learning of Compositional Generalization in NLIcited by 4
  9. 2025A survey of diversity quantification in natural language processing: The why, what, where and howcited by 4
  10. 2023Interpretation modeling: Social grounding of sentences by reasoning over their implicit moral judgmentscited by 2
  11. 2026An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shiftcited by 2
  12. 2023For Generated Text, Is NLI-Neutral Text the Best Text?cited by 1
  13. 2023Pragmatically Appropriate Diversity for Dialogue Evaluationcited by 1
  14. 2026Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolutioncited by 1
  15. 2026Measuring Dataset Diversity from a Geometric Perspectivecited by 0
  16. 2026Towards Scalable Data Diversification for Language Model Pretraining via Leverage Score Samplingcited by 0