mlsanityData to 7 Oct 2026

Comparison of Diverse Decoding Methods from Conditional Language Models

arXiv 1906.06362 ↗First posted 14 Jun 2019cs.CLcited by 78

Not on the map. Why?

Abstract (arXiv)

While conditional language models have greatly improved in their ability to output high-quality natural language, many NLP applications benefit from being able to generate a diverse set of candidate sequences. Diverse decoding strategies aim to, within a given-sized candidate list, cover as much of the space of high-quality outputs as possible, leading to improvements for tasks that re-rank and combine candidate outputs. Standard decoding methods, such as beam search, optimize for generating high likelihood sequences rather than diverse ones, though recent work has focused on increasing diversity in these methods. In this work, we perform an extensive survey of decoding-time strategies for generating diverse outputs from conditional language models. We also show how diversity can be improved without sacrificing quality by over-sampling additional candidates, then filtering to the desired number.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 11; 2023: 11; 2024: 16; 2025: 19; 2026: 9.

78 papers.

Sort:

By area:

  1. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 153
  2. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  3. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  4. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  5. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 112
  6. 2024ARGS: Alignment as Reward-Guided Searchcited by 99
  7. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 97
  8. 2022On Decoding Strategies for Neural Text Generatorscited by 89
  9. 2021Sparks: Inspiration for Science Writing using Language Modelscited by 80
  10. 2025Jointly Reinforcing Diversity and Quality in Language Model Generationscited by 79
  11. 2024ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reorderingcited by 50
  12. 2025Modifying Large Language Model Post-Training for Diverse Creative Writingcited by 39
38 more
  1. 2024Improving Uncertainty Estimation through Semantically Diverse Language Generationcited by 34
  2. 2023Closing the Curious Case of Neural Text Degenerationcited by 30
  3. 2021Diversifying Dialog Generation via Adaptive Label Smoothingcited by 25
  4. 2022Why is constrained neural language generation particularly challenging?cited by 19
  5. 2021ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learningcited by 18
  6. 2023Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Surveycited by 18
  7. 2024Permute-and-Flip: An optimally stable and watermarkable decoder for LLMscited by 17
  8. 2024Evaluating Diversity in Automatic Poetry Generationcited by 17
  9. 2024GeAR: Graph-enhanced Agent for Retrieval-augmented Generationcited by 17
  10. 2022On the probability-quality paradox in language generationcited by 16
  11. 2021A New Approach to Overgenerating and Scoring Abstractive Summariescited by 13
  12. 2024Not Just Novelty: A Longitudinal Study on Utility and Customization of an AI Workflowcited by 13
  13. 2023PathFinder: Guided Search over Multi-Step Reasoning Pathscited by 12
  14. 2023Tweetorial Hooks: Generative AI Tools to Motivate Science on Social Mediacited by 11
  15. 2024Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decodingcited by 11
  16. 2021Diverse and Specific Clarification Question Generation with Keywordscited by 10
  17. 2025Efficient Beam Search for Large Language Models Using Trie-Based Decodingcited by 10
  18. 2023Generating EDU Extracts for Plan-Guided Summary Re-Rankingcited by 9
  19. 2024Guiding and Diversifying LLM-Based Story Generation via Answer Set Programmingcited by 9
  20. 2025Reference-free Evaluation Metrics for Text Generation: A Surveycited by 9
  21. 2025When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMscited by 9
  22. 2025Asking a Language Model for Diverse Responsescited by 9
  23. 2021Empathetic BERT2BERT Conversational Model: Learning Arabic Language Generation with Little Datacited by 7
  24. 2025What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversitycited by 7
  25. 2023A Methodology for Generative Spelling Correction via Natural Spelling Errors Emulation across Multiple Domains and Languagescited by 5
  26. 2024SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generationcited by 5
  27. 2021Sentence-Permuted Paragraph Generationcited by 4
  28. 2022ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translationcited by 4
  29. 2024ConvoSense: Overcoming Monotonous Commonsense Inferences for Conversational AIcited by 4
  30. 2024Representation Bias of Adolescents in AI: A Bilingual, Bicultural Studycited by 4
  31. 2025A survey of diversity quantification in natural language processing: The why, what, where and howcited by 4
  32. 2024Quasi-random Multi-Sample Inference for Large Language Modelscited by 3
  33. 2025Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Modelscited by 3
  34. 2021A Single Example Can Improve Zero-Shot Data Generationcited by 2
  35. 2021Towards More Robust Natural Language Understandingcited by 2
  36. 2022Belief Revision based Caption Re-ranker with Visual Semantic Informationcited by 2
  37. 2023Comparing Apples to Apples: Generating Aspect-Aware Comparative Sentences from User Reviewscited by 2
  38. 2023Interpretation modeling: Social grounding of sentences by reasoning over their implicit moral judgmentscited by 2

50 most cited of 78 All 78 (JSON)