# Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots

arXiv: https://arxiv.org/abs/1809.01941 · HTML: https://mlsanity.com/paper/1809.01941/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 12 of 12)

- [Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer](https://mlsanity.com/paper/2204.03638/) (2022)
- [Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech](https://mlsanity.com/paper/2106.01625/) (2021)
- [Diversifying Dialog Generation via Adaptive Label Smoothing](https://mlsanity.com/paper/2105.14556/) (2021)
- [A Simple Contrastive Learning Objective for Alleviating Neural Text Degeneration](https://mlsanity.com/paper/2205.02517/) (2022)
- [DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations](https://mlsanity.com/paper/2306.16770/) (2023)
- [RetroDiff: Retrosynthesis as Multi-stage Distribution Interpolation](https://mlsanity.com/paper/2311.14077/) (2023)
- [Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems](https://mlsanity.com/paper/2105.00079/) (2021)
- [Augmenting Compliance-Guaranteed Customer Service Chatbots: Context-Aware Knowledge Expansion with Large Language Models](https://mlsanity.com/paper/2410.12444/) (2024)
- [Language Model Evaluation in Open-ended Text Generation](https://mlsanity.com/paper/2108.03578/) (2021)
- [Generate, Evaluate, and Select: A Dialogue System with a Response Evaluator for Diversity-Aware Response Generation](https://mlsanity.com/paper/2206.04937/) (2022)
- [From Persona to Person: Enhancing the Naturalness with Multiple Discourse Relations Graph Learning in Personalized Dialogue Generation](https://mlsanity.com/paper/2506.11557/) (2025)
- [DIVE: Towards Descriptive and Diverse Visual Commonsense Generation](https://mlsanity.com/paper/2408.08021/) (2024)

## Abstract (arXiv)

Diversity is a long-studied topic in information retrieval that usually refers to the requirement that retrieved results should be non-repetitive and cover different aspects. In a conversational setting, an additional dimension of diversity matters: an engaging response generation system should be able to output responses that are diverse and interesting. Sequence-to-sequence (Seq2Seq) models have been shown to be very effective for response generation. However, dialogue responses generated by Seq2Seq models tend to have low diversity. In this paper, we review known sources and existing approaches to this low-diversity problem. We also identify a source of low diversity that has been little studied so far, namely model over-confidence. We sketch several directions for tackling model over-confidence and, hence, the low-diversity problem, including confidence penalties and label smoothing.