mlsanityData to 7 Oct 2026

Quality-Aware Decoding for Neural Machine Translation

arXiv 2205.00978 ↗First posted 2 May 2022cs.CLcited by 49

Not on the map. Why?

Abstract (arXiv)

Despite the progress in machine translation quality estimation and evaluation in the last years, decoding in neural machine translation (NMT) is mostly oblivious to this and centers around finding the most probable translation according to the model (MAP decoding), approximated with beam search. In this paper, we bring together these two lines of research and propose quality-aware decoding for NMT, by leveraging recent breakthroughs in reference-free and reference-based MT evaluation through various inference methods like N-best reranking and minimum Bayes risk decoding. We perform an extensive comparison of various possible candidate generation and ranking methods across four datasets and two model classes and find that quality-aware decoding consistently outperforms MAP-based decoding according both to state-of-the-art automatic metrics (COMET and BLEURT) and to human assessments. Our code is available at https://github.com/deep-spin/qaware-decode.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 44 of its 81 references that are in the map. First posted on arXiv 2 May 2022.

Builds on

44 of 81 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  9. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  10. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  11. 2018Hierarchical Neural Story Generationcited by 1,182
  12. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
32 more
  1. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  2. 2019When Does Label Smoothing Help?cited by 985
  3. 2017Six Challenges for Neural Machine Translationcited by 334
  4. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  5. 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
  6. 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 161
  7. 2020Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metricscited by 155
  8. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  9. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  10. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
  11. 2018Correcting Length Bias in Neural Machine Translationcited by 119
  12. 2015Minimum Risk Training for Neural Machine Translationcited by 114
  13. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  14. 2020If beam search is the answer, what was the question?cited by 114
  15. 2020Rethinking embedding coupling in pre-trained language modelscited by 103
  16. 2021Learning Compact Metrics for MTcited by 93
  17. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  18. 2020TransQuest: Translation Quality Estimation with Cross-lingual Transformerscited by 59
  19. 2020Unbabel's Participation in the WMT20 Metrics Shared Taskcited by 56
  20. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  21. 2017Classical Structured Prediction Losses for Sequence to Sequence Learningcited by 55
  22. 2021Machine Translation Decoding beyond Beam Searchcited by 55
  23. 2019OpenKiwi: An Open Source Framework for Quality Estimationcited by 49
  24. 2020Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Modelscited by 44
  25. 2022Identifying Weaknesses in Machine Translation Metrics Through Minimum Bayes Risk Decoding: A Case Study for COMETcited by 41
  26. 2019Simple and Effective Noisy Channel Modeling for Neural Machine Translationcited by 37
  27. 2021Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 32
  28. 2018Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translationcited by 29
  29. 2016The Neural Noisy Channelcited by 19
  30. 2016Neural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Latticescited by 17
  31. 2017Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translationcited by 8
  32. 2021What Do You Get When You Cross Beam Search with Nucleus Sampling?cited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 20; 2024: 11; 2025: 7; 2026: 2.

49 papers.

Sort:

By area:

  1. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  2. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  3. 2022Calibrating Sequence likelihood Improves Conditional Language Generationcited by 121
  4. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  5. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  6. 2023Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibrationcited by 71
  7. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  8. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 60
  9. 2023Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translationcited by 57
  10. 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generationcited by 55
  11. 2022LENS: A Learnable Evaluation Metric for Text Simplificationcited by 54
  12. 2022On the Limitations of Reference-Free Evaluations of Generated Textcited by 52
37 more
  1. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
  2. 2023What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variabilitycited by 51
  3. 2023It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Riskcited by 49
  4. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  5. 2023MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methodscited by 33
  6. 2025GRAM: A Generative Foundation Reward Model for Reward Generalizationcited by 25
  7. 2024Non-Exchangeable Conformal Language Generation with Nearest Neighborscited by 21
  8. 2024Better Instruction-Following Through Minimum Bayes Riskcited by 19
  9. 2023An Empirical Study of Translation Hypothesis Ensembling with Large Language Modelscited by 16
  10. 2023Towards Explainable Evaluation Metrics for Machine Translationcited by 15
  11. 2023Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Referencecited by 11
  12. 2023ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkitcited by 10
  13. 2023Model-Based Minimum Bayes Risk Decoding for Text Generationcited by 10
  14. 2025Uncertainty-Aware Decoding with Minimum Bayes Riskcited by 10
  15. 2023Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Modelcited by 9
  16. 2025When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMscited by 9
  17. 2023Language Model Decoding as Direct Metrics Optimizationcited by 8
  18. 2022Best-k Search Algorithm for Neural Text Generationcited by 6
  19. 2023Learning Evaluation Models from Large Language Models for Sequence Generationcited by 6
  20. 2023Quantifying the Plausibility of Context Reliance in Neural Machine Translationcited by 6
  21. 2025Adding Chocolate to Mint: Mitigating Metric Interference in Machine Translationcited by 6
  22. 2022Adam Mickiewicz University at WMT 2022: NER-Assisted and Quality-Aware Neural Machine Translationcited by 5
  23. 2023EEL: Efficiently Encoding Lattices for Rerankingcited by 4
  24. 2023Calibrating Likelihoods towards Consistency in Summarization Modelscited by 4
  25. 2024DOCE: Finding the Sweet Spot for Execution-Based Code Generationcited by 4
  26. 2024Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysiscited by 4
  27. 2022Improving abstractive summarization with energy-based re-rankingcited by 3
  28. 2024Reranking Laws for Language Generation: A Communication-Theoretic Perspectivecited by 3
  29. 2024Learning from others' mistakes: Finetuning machine translation models with span-level error annotationscited by 3
  30. 2025Investigating Test-Time Scaling with Reranking for Machine Translationcited by 3
  31. 2025Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Modelscited by 3
  32. 2024Findings of the WMT 2024 Shared Task on Chat Translationcited by 2
  33. 2024A Context-aware Framework for Translation-mediated Conversationscited by 2
  34. 2025Calibrating Translation Decoding with Quality Estimation on LLMscited by 2
  35. 2022A Simple Baseline for Beam Search Rerankingcited by 0
  36. 2026FlashEvaluator: Expanding Search Space with Parallel Sequence-Level Evaluationcited by 0
  37. 2026Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypothesescited by 0