mlsanityData to 7 Oct 2026

A Simple, Fast Diverse Decoding Algorithm for Neural Generation

arXiv 1611.08562 ↗First posted 25 Nov 2016cs.CLcited by 58

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a simple, fast decoding algorithm that fosters diversity in neural generation. The algorithm modifies the standard beam search algorithm by adding an inter-sibling ranking penalty, favoring choosing hypotheses from diverse parents. We evaluate the proposed model on the tasks of dialogue response generation, abstractive summarization and machine translation. We find that diverse decoding helps across all tasks, especially those for which reranking is needed. We further propose a variation that is capable of automatically adjusting its diversity decoding rates for different inputs using reinforcement learning (RL). We observe a further performance boost from this RL technique. This paper includes material from the unpublished script "Mutual Information and Diverse Decoding Improve Neural Machine Translation" (Li and Jurafsky, 2016).

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 15; 2023: 6; 2024: 7; 2025: 10; 2026: 3.

58 papers.

Sort:

By area:

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  3. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  4. 2022Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generationcited by 90
  5. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  6. 2024DeAL: Decoding-time Alignment for Large Language Modelscited by 55
  7. 2024Forcing Diffuse Distributions out of Language Modelscited by 48
  8. 2025Modifying Large Language Model Post-Training for Diverse Creative Writingcited by 39
  9. 2022Graph-based Spatial Transformer with Memory Replay for Multi-future Pedestrian Trajectory Predictioncited by 36
  10. 2022DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generationcited by 35
  11. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  12. 2024Self-Improvement for Neural Combinatorial Optimization: Sample without Replacement, but Improvementcited by 32
38 more
  1. 2022Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbotscited by 31
  2. 2021Focus Attention: Promoting Faithfulness and Diversity in Summarizationcited by 30
  3. 2023Understanding In-Context Learning from Repetitionscited by 27
  4. 2023An Overview on Language Models: Recent Developments and Outlookcited by 22
  5. 2021Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interactioncited by 21
  6. 2022One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translationcited by 20
  7. 2023Long Horizon Temperature Scalingcited by 18
  8. 2023Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Surveycited by 18
  9. 2021Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translationcited by 17
  10. 2022Measuring and Improving Semantic Diversity of Dialogue Generationcited by 16
  11. 2021multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoningcited by 15
  12. 2021Diversity driven Query Rewriting in Search Advertisingcited by 14
  13. 2024Semantic Sensitivities and Inconsistent Predictions: Measuring the Fragility of NLI Modelscited by 14
  14. 2021ConRPG: Paraphrase Generation using Contexts as Regularizercited by 13
  15. 2022Paraphrase Identification with Deep Learning: A Review of Datasets and Methodscited by 13
  16. 2021Mixup Decoding for Diverse Machine Translationcited by 10
  17. 2024To Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translationcited by 10
  18. 2021Towards Efficiently Diversifying Dialogue Generation via Embedding Augmentationcited by 9
  19. 2025Zero-Shot Verification-guided Chain of Thoughtscited by 9
  20. 2025Semantic-guided Diverse Decoding for Large Language Modelcited by 9
  21. 2022What Makes Data-to-Text Generation Hard for Pretrained Language Models?cited by 8
  22. 2021Triangular Bidword Generation for Sponsored Search Auctioncited by 5
  23. 2026Free Lunch for Pass@k? Low Cost Diverse Sampling for Diffusion Language Modelscited by 5
  24. 2021REAMsharp: An Enhancement Approach to Reference-based Evaluation Metrics for Open-domain Dialog Generationcited by 4
  25. 2021RewardsOfSum: Exploring Reinforcement Learning Rewards for Summarisationcited by 4
  26. 2022Prompting for a conversation: How to control a dialog model?cited by 4
  27. 2023Learning Descriptive Image Captioning via Semipermeable Maximum Likelihood Estimationcited by 4
  28. 2023Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialoguecited by 4
  29. 2022Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attentioncited by 3
  30. 2022A Survey on Natural Language Processing for Programmingcited by 3
  31. 2025DIVE: Diversified Iterative Self-Improvementcited by 3
  32. 2021DeepAltTrip: Top-k Alternative Itineraries for Trip Recommendationcited by 2
  33. 2022RMBR: A Regularized Minimum Bayes Risk Reranking Framework for Machine Translationcited by 2
  34. 2025An Ensemble Model with Attention Based Mechanism for Image Captioningcited by 2
  35. 2025Efficient semantic uncertainty quantification in language models via diversity-steered samplingcited by 2
  36. 2026Inducing Sustained Creativity and Diversity in Large Language Modelscited by 2
  37. 2021Toward Improving Coherence and Diversity of Slogan Generationcited by 1
  38. 2021A Condense-then-Select Strategy for Text Summarizationcited by 1

50 most cited of 58 All 58 (JSON)