mlsanityData to 7 Oct 2026

Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation

arXiv 2108.04718 ↗First posted 10 Aug 2021cs.CLcited by 63

Not on the map. Why?

Abstract (arXiv)

In NMT we search for the mode of the model distribution to form predictions. The mode and other high-probability translations found by beam search have been shown to often be inadequate in a number of ways. This prevents improving translation quality through better search, as these idiosyncratic translations end up selected by the decoding algorithm, a problem known as the beam search curse. Recently, an approximation to minimum Bayes risk (MBR) decoding has been proposed as an alternative decision rule that would likely not suffer from the same problems. We analyse this approximation and establish that it has no equivalent to the beam search curse. We then design approximations that decouple the cost of exploration from the cost of robust estimation of expected utility. This allows for much larger hypothesis spaces, which we show to be beneficial. We also show that mode-seeking strategies can aid in constructing compact sets of promising hypotheses and that MBR is effective in identifying good translations in them. We conduct experiments on three language pairs varying in amounts of resources available: English into and from German, Romanian, and Nepali.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 28 of its 79 references that are in the map. First posted on arXiv 10 Aug 2021.

Builds on

28 of 79 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  6. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  7. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  8. 2017Six Challenges for Neural Machine Translationcited by 334
  9. 2019Sparse Sequence-to-Sequence Modelscited by 167
  10. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  11. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  12. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
16 more
  1. 2018Correcting Length Bias in Neural Machine Translationcited by 119
  2. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  3. 2020If beam search is the answer, what was the question?cited by 114
  4. 2018On the Impact of Various Types of Noise on Neural Machine Translationcited by 76
  5. 2019Calibration of Encoder Decoder Models for Neural Machine Translationcited by 67
  6. 2019Domain Robustness in Neural Machine Translationcited by 64
  7. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  8. 2021Machine Translation Decoding beyond Beam Searchcited by 55
  9. 2020Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Modelscited by 44
  10. 2016Mutual Information and Diverse Decoding Improve Neural Machine Translationcited by 35
  11. 2016Neural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Latticescited by 17
  12. 2019Leveraging Sentence Similarity in Natural Language Generation: Improving Beam Search using Range Votingcited by 15
  13. 2016Length bias in Encoder Decoder Models and a Case for Global Conditioningcited by 14
  14. 2021Smoothing and Shrinking the Sparse Seq2Seq Search Spacecited by 11
  15. 2017Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translationcited by 8
  16. 2021Searching for Search Errors in Neural Morphological Inflectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 12; 2024: 22; 2025: 14; 2026: 13.

63 papers.

Sort:

By area:

  1. 2022Continuous diffusion for categorical datacited by 202
  2. 2023What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variabilitycited by 51
  3. 2023It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Riskcited by 49
  4. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  5. 2025Evaluation of Best-of-N Sampling Strategies for Language Model Alignmentcited by 38
  6. 2023Faster Minimum Bayes Risk Decoding with Confidence-based Pruningcited by 34
  7. 2023MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methodscited by 33
  8. 2023Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decodingcited by 28
  9. 2024Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignmentcited by 24
  10. 2023Information Value: Measuring Utterance Predictability as Distance from Plausible Alternativescited by 22
  11. 2023A Cheaper and Better Diffusion Language Model with Soft-Masked Noisecited by 21
  12. 2024Linear-time Minimum Bayes Risk Decoding with Reference Aggregationcited by 21
38 more
  1. 2023An Empirical Study of Translation Hypothesis Ensembling with Large Language Modelscited by 16
  2. 2024Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decodingcited by 15
  3. 2024Mitigating Metric Bias in Minimum Bayes Risk Decodingcited by 15
  4. 2024Don't Rank, Combine! Combining Machine Translation Hypotheses Using Quality Estimationcited by 13
  5. 2024Efficient Minimum Bayes Risk Decoding using Low-Rank Matrix Completion Algorithmscited by 13
  6. 2024Improving Minimum Bayes Risk Decoding with Multi-Promptcited by 13
  7. 2024mbrs: A Library for Minimum Bayes Risk Decodingcited by 13
  8. 2026CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decodingcited by 13
  9. 2024Centroid-Based Efficient Minimum Bayes Risk Decodingcited by 12
  10. 2024Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decodingcited by 11
  11. 2025Multilingual Contextualization of Large Language Models for Document-Level Machine Translationcited by 11
  12. 2023Model-Based Minimum Bayes Risk Decoding for Text Generationcited by 10
  13. 2024Introducing the NewsPaLM MBR and QE Dataset: LLM-Generated High-Quality Parallel Data Outperforms Traditional Web-Crawled Datacited by 10
  14. 2024Beyond Correlation: Interpretable Evaluation of Machine Translation Metricscited by 10
  15. 2025Uncertainty-Aware Decoding with Minimum Bayes Riskcited by 10
  16. 2023Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Modelcited by 9
  17. 2025When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMscited by 9
  18. 2024Large Language Models for Dysfluency Detection in Stuttered Speechcited by 8
  19. 2024Diversity Explains Inference Scaling Laws: Through a Case Study of Minimum Bayes Risk Decodingcited by 8
  20. 2022DC-MBR: Distributional Cooling for Minimum Bayesian Risk Decodingcited by 4
  21. 2024Uncertainty-Guided Likelihood Tree Searchcited by 4
  22. 2024DOCE: Finding the Sweet Spot for Execution-Based Code Generationcited by 4
  23. 2023ReSeTOX: Re-learning attention weights for toxicity mitigation in machine translationcited by 3
  24. 2024Annotation-Efficient Language Model Alignment via Diverse and Representative Response Textscited by 3
  25. 2024Quasi-random Multi-Sample Inference for Large Language Modelscited by 3
  26. 2025Theoretical Guarantees for Minimum Bayes Risk Decodingcited by 3
  27. 2024On the True Distribution Approximation of Minimum Bayes-Risk Decodingcited by 2
  28. 2024A Bayesian Optimization Approach to Machine Translation Rerankingcited by 2
  29. 2025Early-Exit and Instant Confidence Translation Quality Estimationcited by 2
  30. 2025Re-evaluating Minimum Bayes Risk Decoding for Automatic Speech Recognitioncited by 2
  31. 2025Agreement-Constrained Probabilistic Minimum Bayes Risk Decodingcited by 2
  32. 2026Task-Awareness Improves LLM Generations and Uncertaintycited by 2
  33. 2026What Does LLM Refinement Actually Improve? A Systematic Study on Document-Level Literary Translationcited by 2
  34. 2025Generative Distribution Prediction: A Unified Approach to Multimodal Learningcited by 1
  35. 2025Case-Based Decision-Theoretic Decoding with Quality Memoriescited by 1
  36. 2025Structure-Conditional Minimum Bayes Risk Decodingcited by 1
  37. 2026ACES: Who Tests the Tests? Leave-One-Out AUC Consistency for Code Generationcited by 1
  38. 2026Uncertainty-Aware Generation and Decision-Making Under Ambiguitycited by 1

50 most cited of 63 All 63 (JSON)