# Diversifying Dialogue Generation with Non-Conversational Text

arXiv: https://arxiv.org/abs/2005.04346 · HTML: https://mlsanity.com/paper/2005.04346/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2020.

## Built on by (most cited first, 21 of 21)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges](https://mlsanity.com/paper/2204.04859/) (2022)
- [Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models](https://mlsanity.com/paper/2407.04121/) (2024)
- [Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech](https://mlsanity.com/paper/2106.01625/) (2021)
- [Learning Fine-grained Fact-Article Correspondence in Legal Cases](https://mlsanity.com/paper/2104.10726/) (2021)
- [Unsupervised Enrichment of Persona-grounded Dialog with Background Stories](https://mlsanity.com/paper/2106.08364/) (2021)
- [Towards Standard Criteria for human evaluation of Chatbots: A Survey](https://mlsanity.com/paper/2105.11197/) (2021)
- [Diversifying Neural Dialogue Generation via Negative Distillation](https://mlsanity.com/paper/2205.02795/) (2022)
- [Diversifying Question Generation over Knowledge Base via External Natural Questions](https://mlsanity.com/paper/2309.14362/) (2023)
- [StyleChat: Learning Recitation-Augmented Memory in LLMs for Stylized Dialogue Generation](https://mlsanity.com/paper/2403.11439/) (2024)
- [Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems](https://mlsanity.com/paper/2105.00079/) (2021)
- [Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection](https://mlsanity.com/paper/2203.11399/) (2022)
- [Dual Task Framework for Improving Persona-grounded Dialogue Dataset](https://mlsanity.com/paper/2202.05435/) (2022)
- [Follow-on Question Suggestion via Voice Hints for Voice Assistants](https://mlsanity.com/paper/2310.17034/) (2023)
- [Enhancing Task-Oriented Dialogues with Chitchat: a Comparative Study Based on Lexical Diversity and Divergence](https://mlsanity.com/paper/2311.14067/) (2023)
- [Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Survey](https://mlsanity.com/paper/2411.09166/) (2024)
- [BHEISR: Nudging from Bias to Balance -- Promoting Belief Harmony by Eliminating Ideological Segregation in Knowledge-based Recommendations](https://mlsanity.com/paper/2307.02797/) (2023)
- [Hallucinations in medical devices](https://mlsanity.com/paper/2508.14118/) (2025)
- [Gacha Decoding: Eliciting Diverse Generations Through Instruction Following](https://mlsanity.com/paper/2610.01382/) (2026)

## Abstract (arXiv)

Neural network-based sequence-to-sequence (seq2seq) models strongly suffer from the low-diversity problem when it comes to open-domain dialogue generation. As bland and generic utterances usually dominate the frequency distribution in our daily chitchat, avoiding them to generate more interesting responses requires complex data filtering, sampling techniques or modifying the training objective. In this paper, we propose a new perspective to diversify dialogue generation by leveraging non-conversational text. Compared with bilateral conversations, non-conversational text are easier to obtain, more diverse and cover a much broader range of topics. We collect a large-scale non-conversational corpus from multi sources including forum comments, idioms and book snippets. We further present a training paradigm to effectively incorporate these text via iterative back translation. The resulting model is tested on two conversational datasets and is shown to produce significantly more diverse responses without sacrificing the relevance with context.