mlsanityData to 7 Oct 2026

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges

arXiv 1907.05019 ↗First posted 11 Jul 2019cs.CL cs.LGcited by 253

Not on the map. Why?

Abstract (arXiv)

We introduce our efforts towards building a universal neural machine translation (NMT) system capable of translating between any language pair. We set a milestone towards this goal by building a single massively multilingual NMT model handling 103 languages trained on over 25 billion examples. Our system demonstrates effective transfer learning ability, significantly improving translation quality of low-resource languages, while keeping high-resource language translation quality on-par with competitive bilingual baselines. We provide in-depth analysis of various aspects of model building that are crucial to achieving quality and practicality in universal NMT. While we prototype a high-quality universal translation system, our extensive empirical analysis exposes issues that need to be further addressed, and we suggest directions for future research.

Key facts. Cited by 253 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 82; 2022: 48; 2023: 43; 2024: 38; 2025: 20; 2026: 21.

253 papers.

Sort:

By area:

  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  3. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  4. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  5. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  6. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  7. 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 278
  8. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  9. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  10. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  11. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  12. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 138
38 more
  1. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 133
  2. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 130
  3. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  4. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  5. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  6. 2021Scaling Laws for Neural Machine Translationcited by 110
  7. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  8. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
  9. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
  10. 2021Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inferencecited by 96
  11. 2021Survey of Low-Resource Machine Translationcited by 92
  12. 2021Larger-Scale Transformers for Multilingual Masked Language Modelingcited by 86
  13. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 86
  14. 2021It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoningcited by 69
  15. 2021SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Trainingcited by 69
  16. 2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
  17. 2021Including Signed Languages in Natural Language Processingcited by 67
  18. 2021Scaling End-to-End Models for Large-Scale Multilingual ASRcited by 61
  19. 2022Do Current Multi-Task Optimization Methods in Deep Learning Even Help?cited by 60
  20. 2022UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Unitscited by 60
  21. 2022Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEscited by 58
  22. 2021Facebook AI WMT21 News Translation Task Submissioncited by 57
  23. 2022Democratizing Neural Machine Translation with OPUS-MTcited by 57
  24. 2022Data Scaling Laws in NMT: The Effect of Noise and Architecturecited by 55
  25. 2023An Open Dataset and Model for Language Identificationcited by 49
  26. 2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
  27. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  28. 2023Steering Large Language Models for Machine Translation with Finetuning and In-Context Learningcited by 48
  29. 2021The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translationcited by 46
  30. 2021Towards Building ASR Systems for the Next Billion Userscited by 45
  31. 2021Multilingual training for Software Engineeringcited by 43
  32. 2022A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methodscited by 42
  33. 2021IndicBART: A Pre-trained Model for Indic Natural Language Generationcited by 39
  34. 2021The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translationcited by 38
  35. 2021Fast Nearest Neighbor Machine Translationcited by 37
  36. 2023GlotLID: Language Identification for Low-Resource Languagescited by 37
  37. 2023Self-Influence Guided Data Reweighting for Language Model Pre-trainingcited by 37
  38. 2021BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translationcited by 35

50 most cited of 253 All 253 (JSON)