mlsanityData to 7 Oct 2026

Understanding and Mitigating the Uncertainty in Zero-Shot Translation

arXiv 2205.10068 ↗First posted 20 May 2022cs.CL cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Zero-shot translation is a promising direction for building a comprehensive multilingual neural machine translation~(MNMT) system. However, its quality is still not satisfactory due to off-target issues. In this paper, we aim to understand and alleviate the off-target issues from the perspective of uncertainty in zero-shot translation. By carefully examining the translation output and model confidence, we identify two uncertainties that are responsible for the off-target issues, namely, extrinsic data uncertainty and intrinsic model uncertainty. Based on the observations, we propose two lightweight and complementary approaches to denoise the training data for model training and explicitly penalize the off-target translations by unlikelihood training during model training. Extensive experiments on both balanced and imbalanced datasets show that our approaches significantly improve the performance of zero-shot translation over strong MNMT baselines.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 32 of its 48 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

32 of 48 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  4. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  5. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  6. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  7. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  8. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  9. 2016Using the Output Embedding to Improve Language Modelscited by 266
  10. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  11. 2019Massively Multilingual Neural Machine Translationcited by 246
  12. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
20 more
  1. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
  2. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  3. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  4. 2016Edinburgh Neural Machine Translation Systems for WMT 16cited by 76
  5. 2019Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlationscited by 61
  6. 2020Cross-lingual Retrieval for Iterative Self-Supervised Trainingcited by 54
  7. 2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
  8. 2020Improving Zero-Shot Translation by Disentangling Positional Informationcited by 35
  9. 2021Improving Multilingual Translation by Representation and Gradient Regularizationcited by 31
  10. 2021Language Tags Matter for Zero-Shot Neural Machine Translationcited by 29
  11. 2016Neural Machine Translation with Reconstructioncited by 28
  12. 2020Rethinking the Value of Transformer Componentscited by 24
  13. 2019Consistency by Agreement in Zero-shot Neural Machine Translationcited by 22
  14. 2020Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translationcited by 17
  15. 2021Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translationcited by 17
  16. 2021Language Models are Good Translatorscited by 16
  17. 2021Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variablescited by 14
  18. 2021On the Language Coverage Bias for Neural Machine Translationcited by 12
  19. 2021Efficient Inference for Multilingual Neural Machine Translationcited by 10
  20. 2022Focus on the Target's Vocabulary: Masked Label Smoothing for Machine Translationcited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 4; 2025: 2.

9 papers.

  1. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  2. 2022Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languagescited by 13
  3. 2023Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translationcited by 2
  4. 2024Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Featurescited by 2
  5. 2025Uncertainty Quantification for Evaluating Machine Translation Biascited by 2
  6. 2024LCS: A Language Converter Strategy for Zero-Shot Neural Machine Translationcited by 1
  7. 2025PRM-Free Security Alignment of Large Models via Red Teaming and Adversarial Trainingcited by 1
  8. 2024NLIP_Lab-IITH Low-Resource MT System for WMT24 Indic MT Shared Taskcited by 0
  9. 2024On the Shortcut Learning in Multilingual Neural Machine Translationcited by 0