mlsanityData to 14 Oct 2026

On Decoding Strategies for Neural Text Generators

arXiv 2203.15721 ↗First posted 29 Mar 2022cs.CL cs.AIcited by 90

Not on the map. Why?

Abstract (arXiv)

When generating text from probabilistic models, the chosen decoding strategy has a profound effect on the resulting text. Yet the properties elicited by various decoding strategies do not always transfer across natural language generation tasks. For example, while mode-seeking methods like beam search perform remarkably well for machine translation, they have been observed to lead to incoherent and repetitive text in story generation. Despite such observations, the effectiveness of decoding strategies is often assessed with respect to only a single task. This work -- in contrast -- provides a comprehensive analysis of the interaction between language generation tasks and decoding strategies. Specifically, we measure changes in attributes of generated text as a function of both decoding strategy and task using human and automatic evaluation. Our results reveal both previously-observed and surprising findings. For example, the nature of the diversity-quality trade-off in language generation is very task-specific; the length bias often attributed to beam search is not constant across tasks.

Key facts. Cited by 90 papers in the map (data through 14 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

33 of 47 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  8. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  9. 2018Hierarchical Neural Story Generationcited by 1,193
  10. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  11. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  12. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
21 more
  1. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  2. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  3. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 396
  4. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  5. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  6. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  7. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 179
  8. 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
  9. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 138
  10. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
  11. 2018Correcting Length Bias in Neural Machine Translationcited by 120
  12. 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 116
  13. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  14. 2020Trading Off Diversity and Quality in Natural Language Generationcited by 96
  15. 2019Comparison of Diverse Decoding Methods from Conditional Language Modelscited by 78
  16. 2020Consistency of a Recurrent Language Model With Respect to Incomplete Decodingcited by 54
  17. 2017Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Modelscited by 42
  18. 2016Mutual Information and Diverse Decoding Improve Neural Machine Translationcited by 35
  19. 2020A Systematic Characterization of Sampling Algorithms for Open-ended Language Generationcited by 28
  20. 2020Decoding Methods for Neural Narrative Generationcited by 18
  21. 2016Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generationcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 10; 2024: 32; 2025: 25; 2026: 18.

90 papers.

Sort:

By area:

  1. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  2. 2023Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraintscited by 174
  3. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 150
  4. 2024Token-level Direct Preference Optimizationcited by 135
  5. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 114
  6. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 55
  7. 2023What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variabilitycited by 51
  8. 2023It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Riskcited by 51
  9. 2024Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scorescited by 50
  10. 2024Detection and Measurement of Syntactic Templates in Generated Textcited by 44
  11. 2024AI and the Problem of Knowledge Collapsecited by 34
  12. 2024Towards Uncertainty-Aware Language Agentcited by 32
38 more
  1. 2024Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learningcited by 27
  2. 2024Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generationcited by 26
  3. 2024GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulationcited by 23
  4. 2024Permute-and-Flip: An optimally stable and watermarkable decoder for LLMscited by 22
  5. 2024Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challengescited by 18
  6. 2023On the Efficacy of Sampling Adapterscited by 17
  7. 2024Evaluating Diversity in Automatic Poetry Generationcited by 17
  8. 2022On the probability-quality paradox in language generationcited by 16
  9. 2023A Frustratingly Simple Decoding Method for Neural Text Generationcited by 16
  10. 2024Preference Optimization with Multi-Sample Comparisonscited by 16
  11. 2025Top-Down Semantic Refinement for Image Captioningcited by 16
  12. 2024Generalizing Alignment Paradigm of Text-to-Image Generation with Preferences through f-divergence Minimizationcited by 15
  13. 2024Supervised Fine-Tuning as Inverse Reinforcement Learningcited by 12
  14. 2024Dynamic Stochastic Decoding Strategy for Open-Domain Dialogue Generationcited by 10
  15. 2025GUARD: Glocal Uncertainty-Aware Robust Decoding for Effective and Efficient Open-Ended Text Generationcited by 10
  16. 2024Quantifying Generative Media Bias with a Corpus of Real-world and Generated News Articlescited by 9
  17. 2024Towards Better Open-Ended Text Generation: A Multicriteria Evaluation Frameworkcited by 9
  18. 2024Growing a Tail: Increasing Output Diversity in Large Language Modelscited by 9
  19. 2025Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generationcited by 8
  20. 2024Using LLMs to Model the Beliefs and Preferences of Targeted Populationscited by 7
  21. 2024Decoding Game: On Minimax Optimality of Heuristic Text Generation Strategiescited by 7
  22. 2024A Large Sensor Foundation Model Pretrained on Continuous Glucose Monitor Data for Diabetes Managementcited by 6
  23. 2022Language Model Decoding as Likelihood-Utility Alignmentcited by 5
  24. 2024Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedbackcited by 5
  25. 2025Proximalized Preference Optimization for Diverse Feedback Types: A Decomposed Perspective on DPOcited by 5
  26. 2024Uncertainty-Guided Likelihood Tree Searchcited by 4
  27. 2024On A Scale From 1 to 5: Quantifying Hallucination in Faithfulness Evaluationcited by 4
  28. 2025Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Modelscited by 4
  29. 2025Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMscited by 4
  30. 2025Finance Language Model Evaluation (FLaME)cited by 4
  31. 2025Breaking the Modality Barrier: Generative Modeling for Accurate Molecule Retrieval from Mass Spectracited by 4
  32. 2022A Continuum of Generation Tasks for Investigating Length Bias and Degenerate Repetitioncited by 3
  33. 2023Learning Answer Generation using Supervision from Automatic Question Answering Evaluatorscited by 3
  34. 2023Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation Systemcited by 3
  35. 2024Graph Guided Question Answer Generation for Procedural Question-Answeringcited by 3
  36. 2025Multi-Amateur Contrastive Decoding for Text Generationcited by 3
  37. 2026On the Credibility of Evaluating LLMs using Survey Questionscited by 3
  38. 2026The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choicescited by 3

50 most cited of 90 All 90 (JSON)