mlsanityData to 7 Oct 2026

Many-to-English Machine Translation Tools, Data, and Pretrained Models

arXiv 2104.00290 ↗First posted 1 Apr 2021cs.CL cs.AI cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

While there are more than 7000 languages in the world, most translation research efforts have targeted a few high-resource languages. Commercial translation systems support only one hundred languages or fewer, and do not make these models available for transfer to low resource languages. In this work, we present useful tools for machine translation research: MTData, NLCodec, and RTG. We demonstrate their usefulness by creating a multilingual neural machine translation model capable of translating from 500 source languages to English. We make this multilingual model readily downloadable and usable as a service, or as a parent model for transfer-learning to even lower-resource languages.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 30 of its 59 references that are in the map. First posted on arXiv 1 Apr 2021.

Builds on

30 of 59 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  10. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  11. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  12. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
18 more
  1. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  2. 2016Using the Output Embedding to Improve Language Modelscited by 266
  3. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  4. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  5. 2018Marian: Fast Neural Machine Translation in C++cited by 202
  6. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
  7. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  8. 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 127
  9. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  10. 2018Tensor2Tensor for Neural Machine Translationcited by 85
  11. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  12. 2017The IIT Bombay English-Hindi Parallel Corpuscited by 74
  13. 2020Balancing Training for Multilingual Neural Machine Translationcited by 71
  14. 2020PMIndia -- A Collection of Parallel Corpora of Languages of Indiacited by 40
  15. 2019Joey NMT: A Minimalist NMT Toolkit for Novicescited by 39
  16. 2017Nematus: a Toolkit for Neural Machine Translationcited by 16
  17. 2018XNMT: The eXtensible Neural Machine Translation Toolkitcited by 14
  18. 2019A Universal Parent Model for Low-Resource Neural Machine Translation Transfercited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 8; 2024: 8; 2025: 6; 2026: 4.

31 papers.

Sort:

By area:

  1. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 149
  2. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  3. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
  4. 2023An Open Dataset and Model for Language Identificationcited by 49
  5. 2023GlotLID: Language Identification for Low-Resource Languagescited by 37
  6. 2023ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languagescited by 33
  7. 2024Goldfish: Monolingual Language Models for 350 Languagescited by 27
  8. 2024URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Basecited by 21
  9. 2024ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Modelscited by 20
  10. 2024EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Modelscited by 16
  11. 2024Low-Resource Machine Translation through Retrieval-Augmented LLM Prompting: A Study on the Mambai Languagecited by 12
  12. 2022Graphemic Normalization of the Perso-Arabic Scriptcited by 6
19 more
  1. 2024Chasing COMET: Leveraging Minimum Bayes Risk Decoding for Self-Improving Machine Translationcited by 6
  2. 2025Ibom NLP: A Step Toward Inclusive Natural Language Processing for Nigeria's Minority Languagescited by 6
  3. 2023Massively Multi-Lingual Event Understanding: Extraction, Visualization, and Searchcited by 5
  4. 2022Multilingual Bidirectional Unsupervised Translation Through Multilingual Finetuning and Back-Translationcited by 3
  5. 2022Checks and Strategies for Enabling Code-Switched Machine Translationcited by 3
  6. 2023SOTASTREAM: A Streaming Approach to Machine Translation Trainingcited by 3
  7. 2024Generating Gender Alternatives in Machine Translationcited by 3
  8. 2025Token-level Ensembling of Models with Different Vocabulariescited by 3
  9. 2025Low-resource Machine Translation for Code-switched Kazakh-Russian Language Paircited by 3
  10. 2022Detecting Languages Unintelligible to Multilingual Models through Local Structure Probescited by 2
  11. 2025What Do Indonesians Really Need from Language Technology? A Nationwide Surveycited by 2
  12. 2023OpusCleaner and OpusTrainer, open source toolkits for training Machine Translation and Large language modelscited by 1
  13. 2024FAME-MT Dataset: Formality Awareness Made Easy for Machine Translation Purposescited by 1
  14. 2026AfriSUD: A Dependency Treebank Collection for Evaluating Models on African Languagescited by 1
  15. 2025Sunflower: A New Approach To Expanding Coverage of African Languages in Large Language Modelscited by 0
  16. 2025Segmentation Beyond Defaults: Asymmetrical Byte Pair Encoding for Optimal Machine Translation Performancecited by 0
  17. 2026YoNER: A New Yor`ub'a Multi-domain Named Entity Recognition Datasetcited by 0
  18. 2026Analysis of Numerical Localisation in LLM Translationscited by 0
  19. 2026The Interlingua Hypothesis: LLMs Translate via a Latent Task-agnostic Feature Spacecited by 0