# Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation

arXiv: https://arxiv.org/abs/1808.09582 · HTML: https://mlsanity.com/paper/1808.09582/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2018.

## Built on by (most cited first, 29 of 29)

- [Calibrating Sequence likelihood Improves Conditional Language Generation](https://mlsanity.com/paper/2210.00045/) (2022)
- [Sequence-to-Sequence Knowledge Graph Completion and Question Answering](https://mlsanity.com/paper/2203.10321/) (2022)
- [Quality-Aware Decoding for Neural Machine Translation](https://mlsanity.com/paper/2205.00978/) (2022)
- [Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation](https://mlsanity.com/paper/2106.08942/) (2021)
- [Improving Multilingual Translation by Representation and Gradient Regularization](https://mlsanity.com/paper/2109.04778/) (2021)
- [Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks](https://mlsanity.com/paper/2105.00573/) (2021)
- [Reweighting Augmented Samples by Minimizing the Maximal Expected Loss](https://mlsanity.com/paper/2103.08933/) (2021)
- [PathFinder: Guided Search over Multi-Step Reasoning Paths](https://mlsanity.com/paper/2312.05180/) (2023)
- [Smoothing and Shrinking the Sparse Seq2Seq Search Space](https://mlsanity.com/paper/2103.10291/) (2021)
- [A Call for Clarity in Beam Search: How It Works and When It Stops](https://mlsanity.com/paper/2204.05424/) (2022)
- [Eliciting Knowledge from Large Pre-Trained Models for Unsupervised Knowledge-Grounded Conversation](https://mlsanity.com/paper/2211.01587/) (2022)
- [Look-back Decoding for Open-Ended Text Generation](https://mlsanity.com/paper/2305.13477/) (2023)
- [Multi-Sentence Resampling: A Simple Approach to Alleviate Dataset Length Bias and Beam-Search Degradation](https://mlsanity.com/paper/2109.06253/) (2021)
- [DC-MBR: Distributional Cooling for Minimum Bayesian Risk Decoding](https://mlsanity.com/paper/2212.04205/) (2022)
- [When are likely answers right? On Sequence Probability and Correctness in LLMs](https://mlsanity.com/paper/2606.27359/) (2026)
- [GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decoding](https://mlsanity.com/paper/2311.08191/) (2023)
- [Searching for Search Errors in Neural Morphological Inflection](https://mlsanity.com/paper/2102.08424/) (2021)
- [Digging Errors in NMT: Evaluating and Understanding Model Errors from Partial Hypothesis Space](https://mlsanity.com/paper/2106.15217/) (2021)
- [The Implicit Length Bias of Label Smoothing on Beam Search Decoding](https://mlsanity.com/paper/2205.00659/) (2022)
- [On the Depth between Beam Search and Exhaustive Search for Text Generation](https://mlsanity.com/paper/2308.13696/) (2023)
- [Utility-Probability Duality of Neural Networks](https://mlsanity.com/paper/2305.14859/) (2023)
- [Increasing LLM response trustworthiness using voting ensembles](https://mlsanity.com/paper/2510.04048/) (2025)
- [Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees](https://mlsanity.com/paper/2604.20500/) (2026)
- [Bring More Attention to Syntactic Symmetry for Automatic Postediting of High-Quality Machine Translations](https://mlsanity.com/paper/2305.10557/) (2023)
- [Beyond Distribution Sharpening: The Importance of Task Rewards](https://mlsanity.com/paper/2604.16259/) (2026)
- [LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation](https://mlsanity.com/paper/2607.04270/) (2026)
- [SpecCal: Ambiguity-Aware Candidate Calibration for Infrared Spectrum-Based Molecular Structure Reconstruction](https://mlsanity.com/paper/2607.27788/) (2026)
- [Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Models](https://mlsanity.com/paper/2608.14797/) (2026)
- [Beam Search, Self-Consistency, and the Limits of Inference-Time Scaling for Grammar-Constrained Text-to-SQL in Small Language Models](https://mlsanity.com/paper/2608.25761/) (2026)

## Abstract (arXiv)

Beam search is widely used in neural machine translation, and usually improves translation quality compared to greedy search. It has been widely observed that, however, beam sizes larger than 5 hurt translation quality. We explain why this happens, and propose several methods to address this problem. Furthermore, we discuss the optimal stopping criteria for these methods. Results show that our hyperparameter-free methods outperform the widely-used hyperparameter-free heuristic of length normalization by +2.0 BLEU, and achieve the best results among all methods on Chinese-to-English translation.