mlsanityData to 7 Oct 2026

The Missing Ingredient in Zero-Shot Neural Machine Translation

arXiv 1903.07091 ↗First posted 17 Mar 2019cs.CL cs.AI cs.LGcited by 59

Not on the map. Why?

Abstract (arXiv)

Multilingual Neural Machine Translation (NMT) models are capable of translating between multiple source and target languages. Despite various approaches to train such models, they have difficulty with zero-shot translation: translating between language pairs that were not together seen during training. In this paper we first diagnose why state-of-the-art multilingual NMT models that rely purely on parameter sharing, fail to generalize to unseen language pairs. We then propose auxiliary losses on the NMT encoder that impose representational invariance across languages. Our simple approach vastly improves zero-shot translation quality without regressing on supervised directions. For the first time, on WMT14 English-FrenchGerman, we achieve zero-shot performance that is on par with pivoting. We also demonstrate the easy scalability of our approach to multiple languages on the IWSLT 2017 shared task.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 14; 2022: 17; 2023: 16; 2024: 7; 2025: 5.

59 papers.

Sort:

By area:

  1. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 133
  2. 2023Self-Supervised Multimodal Learning: A Surveycited by 58
  3. 2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 55
  4. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
  5. 2022SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representationcited by 32
  6. 2021Language Tags Matter for Zero-Shot Neural Machine Translationcited by 29
  7. 2022Using natural language prompts for machine translationcited by 27
  8. 2022Examining Scaling and Transfer of Language Model Architectures for Machine Translationcited by 26
  9. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMscited by 26
  10. 2021Zero-Shot Cross-lingual Semantic Parsingcited by 19
  11. 2021Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Datacited by 17
  12. 2021A Survey on Low-Resource Neural Machine Translationcited by 16
38 more
  1. 2022T-Modules: Translation Modules for Zero-Shot Cross-Modal Machine Translationcited by 16
  2. 2021Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variablescited by 14
  3. 2022MMTAfrica: Multilingual Machine Translation for African Languagescited by 11
  4. 2023Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularizationcited by 10
  5. 2023Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performancecited by 10
  6. 2025Conditions for Catastrophic Forgetting in Multilingual Translationcited by 10
  7. 2021Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translationcited by 8
  8. 2021Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documentscited by 7
  9. 2022Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translationcited by 7
  10. 2022CrossAligner & Co: Zero-Shot Transfer Methods for Task-Oriented Cross-lingual Natural Language Understandingcited by 7
  11. 2023Neural Machine Translation for the Indigenous Languages of the Americas: An Introductioncited by 7
  12. 2022Adapting to Non-Centered Languages for Zero-shot Multilingual Translationcited by 6
  13. 2022Improving Multilingual Neural Machine Translation System for Indic Languagescited by 6
  14. 2024How Multilingual Are Large Language Models Fine-Tuned for Translation?cited by 6
  15. 2022Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mappingscited by 4
  16. 2022Learning an Artificial Language for Knowledge-Sharing in Multilingual Translationcited by 4
  17. 2023The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translationcited by 4
  18. 2021Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoderscited by 3
  19. 2023Exploring the Impact of Layer Normalization for Zero-shot Neural Machine Translationcited by 3
  20. 2023Gender Lost In Translation: How Bridging The Gap Between Languages Affects Gender Bias in Zero-Shot Multilingual Translationcited by 3
  21. 2024Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translationcited by 3
  22. 2021An Empirical Investigation of Multi-bridge Multilingual NMT modelscited by 2
  23. 2021Multilingual Neural Machine Translation:Can Linguistic Hierarchies Help?cited by 2
  24. 2022Robust Domain Adaptation for Pre-trained Multilingual Neural Machine Translation Modelscited by 2
  25. 2023Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translationcited by 2
  26. 2024Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Featurescited by 2
  27. 2024Language-Informed Beam Search Decoding for Multilingual Machine Translationcited by 2
  28. 2021Self-Learning for Zero Shot Neural Machine Translationcited by 1
  29. 2021Unsupervised Transfer Learning in Multilingual Neural Machine Translation with Cross-Lingual Word Embeddingscited by 1
  30. 2022When do Contrastive Word Alignments Improve Many-to-many Neural Machine Translation?cited by 1
  31. 2022Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translationcited by 1
  32. 2022Informative Language Representation Learning for Massively Multilingual Neural Machine Translationcited by 1
  33. 2022MALM: Mixing Augmented Language Modeling for Zero-Shot Machine Translationcited by 1
  34. 2023Out-of-Distribution Generalization in Text Classification: Past, Present, and Futurecited by 1
  35. 2023TPDM: Selectively Removing Positional Information for Zero-shot Translation via Token-Level Position Disentangle Modulecited by 1
  36. 2023The Effect of Alignment Objectives on Code-Switching Translationcited by 1
  37. 2025Multilingual Non-Autoregressive Machine Translation without Knowledge Distillationcited by 1
  38. 2025Cross-Lingual Stability and Bias in Instruction-Tuned Language Models for Humanitarian NLPcited by 1

50 most cited of 59 All 59 (JSON)