mlsanityData to 7 Oct 2026

If beam search is the answer, what was the question?

arXiv 2010.02650 ↗First posted 6 Oct 2020cs.CLcited by 114

Not on the map. Why?

Abstract (arXiv)

Quite surprisingly, exact maximum a posteriori (MAP) decoding of neural language generators frequently leads to low-quality results. Rather, most state-of-the-art results on language generation tasks are attained using beam search despite its overwhelmingly high search error rate. This implies that the MAP objective alone does not express the properties we desire in text, which merits the question: if beam search is the answer, what was the question? We frame beam search as the exact solution to a different decoding objective in order to gain insights into why high probability under a model alone may not indicate adequacy. We find that beam search enforces uniform information density in text, a property motivated by cognitive science. We suggest a set of decoding objectives that explicitly enforce this property and find that exact decoding with these objectives alleviates the problems encountered when decoding poorly calibrated language generation models. Additionally, we analyze the text produced using various decoding strategies and see that, in our neural machine translation experiments, the extent to which this property is adhered to strongly correlates with BLEU.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 29; 2023: 22; 2024: 15; 2025: 16; 2026: 11.

114 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  3. 2022Offline RL for Natural Language Generation with Implicit Language Q Learningcited by 137
  4. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 136
  5. 2022Locally Typical Samplingcited by 108
  6. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  7. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 102
  8. 2021Sparks: Inspiration for Science Writing using Language Modelscited by 80
  9. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 79
  10. 2023Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agentscited by 66
  11. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 63
  12. 2021Revisiting the Uniform Information Density Hypothesiscited by 63
38 more
  1. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  2. 2021Machine Translation Decoding beyond Beam Searchcited by 55
  3. 2021A Plug-and-Play Method for Controlled Text Generationcited by 55
  4. 2022LENS: A Learnable Evaluation Metric for Text Simplificationcited by 54
  5. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 54
  6. 2021Lower Perplexity is Not Always Human-Likecited by 52
  7. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  8. 2025Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlocited by 45
  9. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  10. 2023GPT-who: An Information Density-based Machine-Generated Text Detectorcited by 38
  11. 2022DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptationcited by 36
  12. 2023Faithfulness-Aware Decoding Strategies for Abstractive Summarizationcited by 31
  13. 2023Closing the Curious Case of Neural Text Degenerationcited by 30
  14. 2022A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generationcited by 28
  15. 2021Why don't people use character-level machine translation?cited by 27
  16. 2024Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative Likelihoodcited by 24
  17. 2021Determinantal Beam Searchcited by 18
  18. 2022On the probability-quality paradox in language generationcited by 16
  19. 2025Top-Down Semantic Refinement for Image Captioningcited by 16
  20. 2023Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSAcited by 15
  21. 2024Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decodingcited by 15
  22. 2025Can Textual Gradient Work in Federated Learning?cited by 14
  23. 2021A New Approach to Overgenerating and Scoring Abstractive Summariescited by 13
  24. 2021FIBER: Fill-in-the-Blanks as a Challenging Video Understanding Evaluation Frameworkcited by 12
  25. 2022Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarizationcited by 12
  26. 2021Smoothing and Shrinking the Sparse Seq2Seq Search Spacecited by 11
  27. 2021A Cognitive Regularizer for Language Modelingcited by 11
  28. 2021Conditional Poisson Stochastic Beam Searchcited by 11
  29. 2024Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decodingcited by 11
  30. 2025Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropycited by 11
  31. 2022A Call for Clarity in Beam Search: How It Works and When It Stopscited by 10
  32. 2022Improved Beam Search for Hallucination Mitigation in Abstractive Summarizationcited by 10
  33. 2023Model-Based Minimum Bayes Risk Decoding for Text Generationcited by 10
  34. 2021Massive-scale Decoding for Text Generation using Latticescited by 8
  35. 2022OTExtSum: Extractive Text Summarisation with Optimal Transportcited by 8
  36. 2022GraphWalks: Efficient Shape Agnostic Geodesic Shortest Path Estimationcited by 8
  37. 2021What Do You Get When You Cross Beam Search with Nucleus Sampling?cited by 7
  38. 2022MAD for Robust Reinforcement Learning in Machine Translationcited by 7

50 most cited of 114 All 114 (JSON)