mlsanityData to 14 Oct 2026

Consistency of a Recurrent Language Model With Respect to Incomplete Decoding

arXiv 2002.02492 ↗First posted 6 Feb 2020cs.LG cs.CL stat.MLcited by 54

Not on the map. Why?

Abstract (arXiv)

Despite strong performance on a variety of tasks, neural sequence models trained with maximum likelihood have been shown to exhibit issues such as length bias and degenerate repetition. We study the related issue of receiving infinite-length sequences from a recurrent language model when using common decoding algorithms. To analyze this issue, we first define inconsistency of a decoding algorithm, meaning that the algorithm can yield an infinite-length sequence that has zero probability under the model. We prove that commonly used incomplete decoding algorithms - greedy search, beam search, top-k sampling, and nucleus sampling - are inconsistent, despite the fact that recurrent language models are trained to produce sequences of finite length. Based on these insights, we propose two remedies which address inconsistency: consistent variants of top-k and nucleus sampling, and a self-terminating recurrent language model. Empirical results show that inconsistency occurs in practice, and that the proposed methods prevent inconsistency.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 9; 2022: 14; 2023: 9; 2024: 12; 2025: 5; 2026: 4.

54 papers.

Sort:

By area:

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  2. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  3. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 268
  4. 2024The pitfalls of next-token predictioncited by 160
  5. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  6. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 137
  7. 2022Truncation Sampling as Language Model Desmoothingcited by 123
  8. 2023Amortizing intractable inference in large language modelscited by 97
  9. 2022Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generationcited by 94
  10. 2022On Decoding Strategies for Neural Text Generatorscited by 90
  11. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 72
  12. 2023Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agentscited by 67
38 more
  1. 2020Text Generation by Learning from Demonstrationscited by 64
  2. 2021Machine Translation Decoding beyond Beam Searchcited by 55
  3. 2022MAUVE Scores for Generative Models: Theory and Practicecited by 36
  4. 2024From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformerscited by 34
  5. 2022A Measure-Theoretic Characterization of Tight Language Modelscited by 33
  6. 2022Reward Gaming in Conditional Text Generationcited by 30
  7. 2021Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematicscited by 27
  8. 2024Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generationcited by 27
  9. 2023Stance Detection: A Practical Guide to Classifying Political Beliefs in Textcited by 20
  10. 2022Evaluating Distributional Distortion in Neural Language Modelingcited by 19
  11. 2022On the probability-quality paradox in language generationcited by 16
  12. 2024Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decodingcited by 15
  13. 2022Knowledge Infused Decodingcited by 12
  14. 2023LongStory: Coherent, Complete and Length Controlled Long story Generationcited by 12
  15. 2021How (Non-)Optimal is the Lexicon?cited by 11
  16. 2023Model-Based Minimum Bayes Risk Decoding for Text Generationcited by 11
  17. 2021Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modelingcited by 10
  18. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 10
  19. 2025GUARD: Glocal Uncertainty-Aware Robust Decoding for Effective and Efficient Open-Ended Text Generationcited by 10
  20. 2025Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanationcited by 9
  21. 2021Extractive and Abstractive Explanations for Fact-Checking and Evaluation of Newscited by 8
  22. 2024Model-based Preference Optimization in Abstractive Summarization without Human Feedbackcited by 7
  23. 2022Best-k Search Algorithm for Neural Text Generationcited by 6
  24. 2024Autoregressive Generation Strategies for Top-K Sequential Recommendationscited by 5
  25. 2021Amortized Noisy Channel Neural Machine Translationcited by 4
  26. 2023Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generationcited by 3
  27. 2024Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Modelscited by 3
  28. 2022Multi-segment preserving sampling for deep manifold samplercited by 2
  29. 2023On the Depth between Beam Search and Exhaustive Search for Text Generationcited by 2
  30. 2025Probability-Consistent Preference Optimization for Enhanced LLM Reasoningcited by 2
  31. 2021Mode recovery in neural autoregressive sequence modelingcited by 1
  32. 2023Mitigating the Exposure Bias in Sentence-Level Grapheme-to-Phoneme (G2P) Transductioncited by 1
  33. 2023On the Representational Capacity of Recurrent Neural Language Modelscited by 1
  34. 2025Energy-Conscious LLM Decoding: Impact of Text Generation Strategies on GPU Energy Consumptioncited by 1
  35. 2025GaLLoP: Gradient-based Sparse Learning on Low-Magnitude Parameterscited by 1
  36. 2022A Non-monotonic Self-terminating Language Modelcited by 0
  37. 2024A Probability--Quality Trade-off in Aligned Language Models and its Relation to Sampling Adaptorscited by 0
  38. 2024Leveraging Customer Feedback for Multi-modal Insight Extractioncited by 0

50 most cited of 54 All 54 (JSON)