mlsanityData to 14 Oct 2026

Analyzing Uncertainty in Neural Machine Translation

arXiv 1803.00047 ↗First posted 28 Feb 2018cs.CLcited by 126

Not on the map. Why?

Abstract (arXiv)

Machine translation is a popular test bed for research in neural sequence-to-sequence models but despite much recent research, there is still a lack of understanding of these models. Practitioners report performance degradation with large beams, the under-estimation of rare words and a lack of diversity in the final translations. Our study relates some of these issues to the inherent uncertainty of the task, due to the existence of multiple valid translations for a single source sentence, and to the extrinsic uncertainty caused by noisy training data. We propose tools and metrics to assess how uncertainty in the data is captured by the model distribution and how it affects search strategies that generate translations. Our results show that search works remarkably well but that models tend to spread too much probability mass over the hypothesis space. Next, we propose tools to assess model calibration and show how to easily fix some shortcomings of current models. As part of this study, we release multiple human reference translations for two popular benchmarks.

Key facts. Cited by 126 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 31; 2022: 30; 2023: 20; 2024: 26; 2025: 15; 2026: 4.

126 papers.

Sort:

By area:

  1. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  3. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  4. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  5. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  6. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  7. 2021Survey of Low-Resource Machine Translationcited by 92
  8. 2022On Decoding Strategies for Neural Text Generatorscited by 90
  9. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 65
  10. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Modelcited by 58
  11. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  12. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
38 more
  1. 2023What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variabilitycited by 51
  2. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  3. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  4. 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
  5. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
  6. 2021A Unified Pruning Framework for Vision Transformerscited by 40
  7. 2021Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Modelscited by 39
  8. 2021Uncertainty-Aware Machine Translation Evaluationcited by 35
  9. 2021Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translationcited by 34
  10. 2022Weakly Supervised Data Augmentation Through Prompting for Dialogue Understandingcited by 32
  11. 2024Towards Uncertainty-Aware Language Agentcited by 32
  12. 2021Distributed NLI: Learning to Predict Human Opinion Distributions for Language Reasoningcited by 31
  13. 2023How to Design Translation Prompts for ChatGPT: An Empirical Studycited by 31
  14. 2021Valid prediction intervals for regression problemscited by 29
  15. 2021Rare Tokens Degenerate All Tokens: Improving Neural Text Generation via Adaptive Gradient Gating for Rare Token Embeddingscited by 27
  16. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  17. 2024Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguitycited by 27
  18. 2024Evaluating language models as risk scorescited by 27
  19. 2022Bootstrapping Multi-view Representations for Fake News Detectioncited by 25
  20. 2022SALTED: A Framework for SAlient Long-Tail Translation Error Detectioncited by 23
  21. 2021Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMTcited by 21
  22. 2022Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Expertscited by 19
  23. 2023Aligning Neural Machine Translation Models: Human Feedback in Training and Inferencecited by 19
  24. 2024Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyondcited by 18
  25. 2021Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translationcited by 17
  26. 2023MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropiescited by 16
  27. 2024QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translationcited by 16
  28. 2021On the Copying Behaviors of Pre-Training for Neural Machine Translationcited by 15
  29. 2022On the Complementarity between Pre-Training and Random-Initialization for Resource-Rich Machine Translationcited by 15
  30. 2025Bag of Tricks for Inference-time Computation of LLM Reasoningcited by 15
  31. 2022Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translationcited by 14
  32. 2024Creative Beam Search: LLM-as-a-Judge For Improving Response Generationcited by 14
  33. 2022Less Learn Shortcut: Analyzing and Mitigating Learning of Spurious Feature-Label Correlationcited by 13
  34. 2022Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languagescited by 13
  35. 2024Don't Rank, Combine! Combining Machine Translation Hypotheses Using Quality Estimationcited by 13
  36. 2024Efficient Minimum Bayes Risk Decoding using Low-Rank Matrix Completion Algorithmscited by 13
  37. 2024mbrs: A Library for Minimum Bayes Risk Decodingcited by 13
  38. 2021How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation?cited by 12

50 most cited of 126 All 126 (JSON)