# A Simple, Fast Diverse Decoding Algorithm for Neural Generation

arXiv: https://arxiv.org/abs/1611.08562 · HTML: https://mlsanity.com/paper/1611.08562/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2016.

## Built on by (most cited first, 30 of 58)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://mlsanity.com/paper/2202.07922/) (2022)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation](https://mlsanity.com/paper/2206.02369/) (2022)
- [Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better](https://mlsanity.com/paper/2212.08597/) (2022)
- [DeAL: Decoding-time Alignment for Large Language Models](https://mlsanity.com/paper/2402.06147/) (2024)
- [Forcing Diffuse Distributions out of Language Models](https://mlsanity.com/paper/2404.10859/) (2024)
- [Modifying Large Language Model Post-Training for Diverse Creative Writing](https://mlsanity.com/paper/2503.17126/) (2025)
- [Graph-based Spatial Transformer with Memory Replay for Multi-future Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2206.05712/) (2022)
- [DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation](https://mlsanity.com/paper/2204.13031/) (2022)
- [The Homogenizing Effect of Large Language Models on Human Expression and Thought](https://mlsanity.com/paper/2508.01491/) (2025)
- [Self-Improvement for Neural Combinatorial Optimization: Sample without Replacement, but Improvement](https://mlsanity.com/paper/2403.15180/) (2024)
- [Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbots](https://mlsanity.com/paper/2209.03463/) (2022)
- [Focus Attention: Promoting Faithfulness and Diversity in Summarization](https://mlsanity.com/paper/2105.11921/) (2021)
- [Understanding In-Context Learning from Repetitions](https://mlsanity.com/paper/2310.00297/) (2023)
- [An Overview on Language Models: Recent Developments and Outlook](https://mlsanity.com/paper/2303.05759/) (2023)
- [Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction](https://mlsanity.com/paper/2106.03044/) (2021)
- [One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation](https://mlsanity.com/paper/2205.14333/) (2022)
- [Long Horizon Temperature Scaling](https://mlsanity.com/paper/2302.03686/) (2023)
- [Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Survey](https://mlsanity.com/paper/2311.09212/) (2023)
- [Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation](https://mlsanity.com/paper/2106.00941/) (2021)
- [Measuring and Improving Semantic Diversity of Dialogue Generation](https://mlsanity.com/paper/2210.05725/) (2022)
- [multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning](https://mlsanity.com/paper/2106.01354/) (2021)
- [Diversity driven Query Rewriting in Search Advertising](https://mlsanity.com/paper/2106.03816/) (2021)
- [Semantic Sensitivities and Inconsistent Predictions: Measuring the Fragility of NLI Models](https://mlsanity.com/paper/2401.14440/) (2024)
- [ConRPG: Paraphrase Generation using Contexts as Regularizer](https://mlsanity.com/paper/2109.00363/) (2021)
- [Paraphrase Identification with Deep Learning: A Review of Datasets and Methods](https://mlsanity.com/paper/2212.06933/) (2022)
- [Mixup Decoding for Diverse Machine Translation](https://mlsanity.com/paper/2109.03402/) (2021)
- [To Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translation](https://mlsanity.com/paper/2401.01419/) (2024)
- [Zero-Shot Verification-guided Chain of Thoughts](https://mlsanity.com/paper/2501.13122/) (2025)

## Abstract (arXiv)

In this paper, we propose a simple, fast decoding algorithm that fosters diversity in neural generation. The algorithm modifies the standard beam search algorithm by adding an inter-sibling ranking penalty, favoring choosing hypotheses from diverse parents. We evaluate the proposed model on the tasks of dialogue response generation, abstractive summarization and machine translation. We find that diverse decoding helps across all tasks, especially those for which reranking is needed. We further propose a variation that is capable of automatically adjusting its diversity decoding rates for different inputs using reinforcement learning (RL). We observe a further performance boost from this RL technique. This paper includes material from the unpublished script "Mutual Information and Diverse Decoding Improve Neural Machine Translation" (Li and Jurafsky, 2016).