mlsanityData to 14 Oct 2026

Building Machine Translation Systems for the Next Thousand Languages

arXiv 2205.03983 ↗First posted 9 May 2022cs.CL cs.AI cs.LGcited by 69

Not on the map. Why?

Abstract (arXiv)

In this paper we share findings from our effort to build practical machine translation (MT) systems capable of translating across over one thousand languages. We describe results in three research domains: (i) Building clean, web-mined datasets for 1500+ languages by leveraging semi-supervised pre-training for language identification and developing data-driven filtering techniques; (ii) Developing practical MT models for under-served languages by leveraging massively multilingual models trained with supervised parallel data for over 100 high-resource languages and monolingual datasets for an additional 1000+ languages; and (iii) Studying the limitations of evaluation metrics for these languages and conducting qualitative analysis of the outputs from our MT models, highlighting several frequent error modes of these types of models. We hope that our work provides useful insights to practitioners working towards building MT systems for currently understudied languages, and highlights research directions that can complement the weaknesses of massively multilingual models in data-sparse settings.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). Builds on 71 of its 150 references that are in the map. First posted on arXiv 9 May 2022.

Builds on

71 of 150 references.

References · 71

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  7. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  8. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  9. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  10. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  11. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  12. 2016Sequence-Level Knowledge Distillationcited by 801
38 more
  1. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  2. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  3. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  4. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  5. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  6. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 347
  7. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  8. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  9. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 273
  10. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  11. 2019Massively Multilingual Neural Machine Translationcited by 246
  12. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
  13. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  14. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  15. 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
  16. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  17. 2017Unsupervised Neural Machine Translationcited by 149
  18. 2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
  19. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  20. 2020A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languagescited by 131
  21. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  22. 2020BLEU might be Guilty but References are not Innocentcited by 110
  23. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
  24. 2018The Best of Both Worlds: Combining Recent Advances in Neural Machine Translationcited by 102
  25. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 98
  26. 2020Extending Multilingual BERT to Low-Resource Languagescited by 88
  27. 2019Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modelingcited by 86
  28. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  29. 2022MAESTRO: Matched Speech Text Representations through Modality Matchingcited by 79
  30. 2020Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpuscited by 77
  31. 2022Unified Speech-Text Pre-training for Speech Translation and Recognitioncited by 64
  32. 2019Generalized Data Augmentation for Low-Resource Translationcited by 53
  33. 2020Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translationcited by 51
  34. 2021Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translationcited by 47
  35. 2016Zero-Resource Translation with Multi-Lingual Neural Machine Translationcited by 46
  36. 2020PMIndia -- A Collection of Parallel Corpora of Languages of Indiacited by 42
  37. 2020Mapping Languages: The Corpus of Global Language Usecited by 42
  38. 2022A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editingcited by 40

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 11; 2023: 25; 2024: 23; 2025: 4; 2026: 6.

69 papers.

Sort:

By area:

  1. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  2. 2023PaLM 2 Technical Reportcited by 1,162
  3. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  4. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  5. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
  6. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 181
  7. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 151
  8. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  9. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  10. 2023Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOMcited by 59
  11. 2024A New Massive Multilingual Dataset for High-Performance Language Technologiescited by 56
  12. 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 56
38 more
  1. 2022Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP Worldcited by 55
  2. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  3. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  4. 2023GlotLID: Language Identification for Low-Resource Languagescited by 39
  5. 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
  6. 2024Goldfish: Monolingual Language Models for 350 Languagescited by 29
  7. 2023Scaling Laws for Multilingual Neural Machine Translationcited by 27
  8. 2023AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languagescited by 24
  9. 2023ChatGPT MT: Competitive for High- (but not Low-) Resource Languagescited by 23
  10. 2025ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilingualitycited by 22
  11. 2024GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languagescited by 16
  12. 2022Multilingual Machine Translation with Hyper-Adapterscited by 15
  13. 2024Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corporacited by 13
  14. 2024Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptionscited by 12
  15. 2025SMOL: Professionally translated parallel data for 115 under-represented languagescited by 12
  16. 2024LexC-Gen: Generating Data for Extremely Low-Resource Languages with Large Language Models and Bilingual Lexiconscited by 11
  17. 2024Back to School: Translation Using Grammar Bookscited by 11
  18. 2024Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruningcited by 10
  19. 2024FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluationcited by 10
  20. 2022Tencent AI Lab - Shanghai Jiao Tong University Low-Resource Translation System for the WMT22 Translation Taskcited by 8
  21. 2023Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translationcited by 8
  22. 2023varepsilon K'U <MASK>: Integrating Yor`ub'a cultural greetings into machine translationcited by 8
  23. 2024Scaling Sign Language Translationcited by 8
  24. 2022When does Parameter-Efficient Transfer Learning Work for Machine Translation?cited by 7
  25. 2022The first neural machine translation system for the Erzya languagecited by 7
  26. 2023Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generationcited by 7
  27. 2022Assessing Digital Language Support on a Global Scalecited by 6
  28. 2024m3P: Towards Multimodal Multilingual Translation with Multimodal Promptcited by 6
  29. 2022Too Brittle To Touch: Comparing the Stability of Quantization and Distillation Towards Developing Lightweight Low-Resource MT Modelscited by 5
  30. 2023Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translationcited by 5
  31. 2024On the Evaluation Practices in Multilingual NLP: Can Machine Translation Offer an Alternative to Human Translations?cited by 5
  32. 2024PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from Related Example Bankscited by 5
  33. 2022Rethinking Round-Trip Translation for Machine Translation Evaluationcited by 4
  34. 2024Triples-to-isiXhosa (T2X): Addressing the Challenges of Low-Resource Agglutinative Data-to-Text Generationcited by 3
  35. 2024Mufu: Multilingual Fused Learning for Low-Resource Translation with LLMcited by 3
  36. 2025Data Augmentation With Back translation for Low Resource languages: A case of English and Lugandacited by 3
  37. 2026Artificial intelligence is creating a new global linguistic hierarchycited by 3
  38. 2023Beyond Arabic: Software for Perso-Arabic Script Manipulationcited by 2

50 most cited of 69 All 69 (JSON)