mlsanityData to 7 Oct 2026

Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models

arXiv 1610.02424 ↗First posted 7 Oct 2016cs.AI cs.CL cs.CVcited by 299

Not on the map. Why?

Cited by 22 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Neural sequence models are widely used to model time-series data. Equally ubiquitous is the usage of beam search (BS) as an approximate inference algorithm to decode output sequences from these models. BS explores the search space in a greedy left-right fashion retaining only the top-B candidates - resulting in sequences that differ only slightly from each other. Producing lists of nearly identical sequences is not only computationally wasteful but also typically fails to capture the inherent ambiguity of complex AI tasks. To overcome this problem, we propose Diverse Beam Search (DBS), an alternative to BS that decodes a list of diverse outputs by optimizing for a diversity-augmented objective. We observe that our method finds better top-1 solutions by controlling for the exploration and exploitation of the search space - implying that DBS is a better search algorithm. Moreover, these gains are achieved with minimal computational or memory over- head as compared to beam search. To demonstrate the broad applicability of our method, we present results on image captioning, machine translation and visual question generation using both standard quantitative metrics and qualitative human studies. Further, we study the role of diversity for image-grounded language generation tasks as the complexity of the image changes. We observe that our method consistently outperforms BS and previously proposed techniques for diverse decoding from neural sequence models.

Key facts. Cited by 299 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 29; 2022: 62; 2023: 39; 2024: 68; 2025: 51; 2026: 49.

299 papers.

Sort:

By area:

  1. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  2. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  3. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  4. 2022Learning Video Representations from Large Language Modelscited by 204
  5. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  6. 2022Translation between Molecules and Natural Languagecited by 173
  7. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
  8. 2021SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarizationcited by 131
  9. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  10. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  11. 2022Calibrating Sequence likelihood Improves Conditional Language Generationcited by 121
  12. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
38 more
  1. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 106
  2. 2022Semantic Probabilistic Layers for Neuro-Symbolic Learningcited by 103
  3. 2023GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretationcited by 101
  4. 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 96
  5. 2023Amortizing intractable inference in large language modelscited by 94
  6. 2022On Decoding Strategies for Neural Text Generatorscited by 89
  7. 2022Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Trainingcited by 88
  8. 2021Sparks: Inspiration for Science Writing using Language Modelscited by 80
  9. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoisecited by 79
  10. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 79
  11. 2025Jointly Reinforcing Diversity and Quality in Language Model Generationscited by 79
  12. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  13. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 72
  14. 2020Dialogue Response Ranking Training with Large-Scale Human Feedback Datacited by 68
  15. 2022Recovering Private Text in Federated Learning of Language Modelscited by 67
  16. 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agentscited by 66
  17. 2021PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domainscited by 61
  18. 2022SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarizationcited by 61
  19. 2021The Transformer Network for the Traveling Salesman Problemcited by 59
  20. 2021Improving Factual Consistency of Abstractive Summarization via Question Answeringcited by 55
  21. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  22. 2024Forcing Diffuse Distributions out of Language Modelscited by 48
  23. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  24. 2021Human-like Controllable Image Captioning with Verb-specific Semantic Rolescited by 41
  25. 2024Measuring memorization in language models via probabilistic extractioncited by 40
  26. 2025Modifying Large Language Model Post-Training for Diverse Creative Writingcited by 39
  27. 2025Weight Ensembling Improves Reasoning in Language Modelscited by 36
  28. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  29. 2024Reward Guided Latent Consistency Distillationcited by 35
  30. 2024Improving Uncertainty Estimation through Semantically Diverse Language Generationcited by 34
  31. 2025JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restorationcited by 32
  32. 2021More but Correct: Generating Diversified and Entity-revised Medical Responsecited by 31
  33. 2023CodeFusion: A Pre-trained Diffusion Model for Code Generationcited by 31
  34. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 29
  35. 2022A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generationcited by 28
  36. 2022Rethinking and Refining the Distinct Metriccited by 27
  37. 2023Aligning Language Models with Human Preferences via a Bayesian Approachcited by 27
  38. 2024Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generationcited by 27

50 most cited of 299 All 299 (JSON)