mlsanityData to 7 Oct 2026

No Language Left Behind: Scaling Human-Centered Machine Translation

arXiv 2207.04672 ↗First posted 11 Jul 2022cs.CL cs.AIcited by 1,290

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

Driven by the goal of eradicating language barriers on a global scale, machine translation has solidified itself as a key focus of artificial intelligence research today. However, such efforts have coalesced around a small subset of languages, leaving behind the vast majority of mostly low-resource languages. What does it take to break the 200 language barrier while ensuring safe, high quality results, all while keeping ethical considerations in mind? In No Language Left Behind, we took on this challenge by first contextualizing the need for low-resource language translation support through exploratory interviews with native speakers. Then, we created datasets and models aimed at narrowing the performance gap between low and high-resource languages. More specifically, we developed a conditional compute model based on Sparsely Gated Mixture of Experts that is trained on data obtained with novel and effective data mining techniques tailored for low-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Critically, we evaluated the performance of over 40,000 different translation directions using a human-translated benchmark, Flores-200, and combined human evaluation with a novel toxicity benchmark covering all languages in Flores-200 to assess translation safety. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art, laying important groundwork towards realizing a universal translation system. Finally, we open source all contributions described in this work, accessible at https://github.com/facebookresearch/fairseq/tree/nllb.

Key facts. Cited by 1,290 papers in the map (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. First posted on arXiv 11 Jul 2022.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 47; 2023: 239; 2024: 450; 2025: 334; 2026: 220.

1,290 papers.

Sort:

By area:

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2023A Survey of Large Language Modelscited by 2,537
  3. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  4. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  5. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  6. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  7. 2023The Falcon Series of Open Language Modelscited by 526
  8. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  9. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  10. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
  11. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  12. 2023Tool Learning with Foundation Modelscited by 302
38 more
  1. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  2. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  3. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  4. 2024A Survey on Mixture of Experts in Large Language Modelscited by 233
  5. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
  6. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
  7. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  8. 2024Many-Shot In-Context Learningcited by 203
  9. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
  10. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
  11. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 180
  12. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 174
  13. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 163
  14. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 156
  15. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 149
  16. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 144
  17. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  18. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  19. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  20. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  21. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  22. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  23. 2023Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Modelscited by 117
  24. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  25. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 111
  26. 2023A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captionscited by 107
  27. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 105
  28. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  29. 2024EuroLLM: Multilingual Language Models for Europecited by 103
  30. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 102
  31. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inferencecited by 101
  32. 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 99
  33. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  34. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 97
  35. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 92
  36. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 92
  37. 2023ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Modelscited by 91
  38. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91

50 most cited of 1,290 All 1,290 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Neural No-Resource Language Translation: A Comparative Evaluation of Approaches
  2. 2024Responsible Multilingual Large Language Models: A Survey of Development, Applications, and Societal Impact
  3. 2026Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation
  4. 2025Low-Resource, High-Impact: Building Corpora for Inclusive Language Technologies
  5. 2025Scaling Low-Resource MT via Synthetic Data Generation with LLMs
  6. 2026MERIT: Multilingual Expert-Reward Informed Tuning for Chinese-Centric Low-Resource Machine Translation
  7. 2026IFMTBench: A Comprehensive Benchmark for Multilingual Translation Instruction Following
  8. 2025Estonian WinoGrande Dataset: Comparative Analysis of LLM Performance on Human and Machine Translation