mlsanityData to 14 Oct 2026

Beyond English-Centric Multilingual Machine Translation

arXiv 2010.11125 ↗First posted 21 Oct 2020cs.CL cs.LGcited by 561

Not on the map. Why?

Abstract (arXiv)

Existing work in translation demonstrated the potential of massively multilingual machine translation by training a single model able to translate between any pair of languages. However, much of this work is English-Centric by training only on data which was translated from or to English. While this is supported by large sources of training data, it does not reflect translation needs worldwide. In this work, we create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Then, we explore how to effectively increase model capacity through a combination of dense scaling and language-specific sparse parameters to create high quality models. Our focus on non-English-Centric models brings gains of more than 10 BLEU when directly translating between non-English directions while performing competitively to the best single systems of WMT. We open-source our scripts so that others may reproduce the data, evaluation, and final M2M-100 model.

Key facts. Cited by 561 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 62; 2022: 111; 2023: 132; 2024: 131; 2025: 75; 2026: 50.

561 papers.

Sort:

By area:

  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  4. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  5. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  6. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 399
  7. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 394
  8. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  9. 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 320
  10. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  11. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 249
  12. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
38 more
  1. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
  2. 2022DeepNet: Scaling Transformers to 1,000 Layerscited by 178
  3. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  4. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 176
  5. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  6. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 160
  7. 2022A Review of Sparse Expert Models in Deep Learningcited by 148
  8. 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
  9. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 132
  10. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  11. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  12. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 109
  13. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  14. 2021The Multilingual TEDx Corpus for Speech Recognition and Translationcited by 101
  15. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 98
  16. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  17. 2021Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inferencecited by 96
  18. 2021Survey of Low-Resource Machine Translationcited by 92
  19. 2023Modular Deep Learningcited by 83
  20. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  21. 2021Scalable and Efficient MoE Training for Multitask Multilingual Modelscited by 76
  22. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 73
  23. 2022Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answeringcited by 72
  24. 2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
  25. 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
  26. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  27. 2021Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translationcited by 62
  28. 2021The Dawn of Quantum Natural Language Processingcited by 61
  29. 2023Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOMcited by 59
  30. 2021Facebook AI WMT21 News Translation Task Submissioncited by 57
  31. 2022AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Modelcited by 57
  32. 2021BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modelingcited by 56
  33. 2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 56
  34. 2022Data Scaling Laws in NMT: The Effect of Noise and Architecturecited by 55
  35. 2021A Survey on Green Deep Learningcited by 52
  36. 2023Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languagescited by 52
  37. 2022PromDA: Prompt-based Data Augmentation for Low-Resource NLU Taskscited by 51
  38. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51

50 most cited of 561 All 561 (JSON)