mlsanityData to 7 Oct 2026

Neural Machine Translation for Low-Resource Languages: A Survey

arXiv 2106.15115 ↗First posted 29 Jun 2021cs.CL cs.AIcited by 108

Not on the map. Why?

Abstract (arXiv)

Neural Machine Translation (NMT) has seen a tremendous spurt of growth in less than ten years, and has already entered a mature phase. While considered as the most widely used solution for Machine Translation, its performance on low-resource language pairs still remains sub-optimal compared to the high-resource counterparts, due to the unavailability of large parallel corpora. Therefore, the implementation of NMT techniques for low-resource language pairs has been receiving the spotlight in the recent NMT research arena, thus leading to a substantial amount of research reported on this topic. This paper presents a detailed survey of research advancements in low-resource language NMT (LRL-NMT), along with a quantitative analysis aimed at identifying the most popular solutions. Based on our findings from reviewing previous work, this survey paper provides a set of guidelines to select the possible NMT technique for a given LRL data setting. It also presents a holistic view of the LRL-NMT research landscape and provides a list of recommendations to further enhance the research efforts on LRL-NMT.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 143 of its 226 references that are in the map. First posted on arXiv 29 Jun 2021.

Builds on

143 of 226 references.

References · 143

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  7. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  8. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  9. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  10. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  11. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  12. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
38 more
  1. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  2. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  3. 2017Word Translation Without Parallel Datacited by 480
  4. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  5. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  6. 2017Six Challenges for Neural Machine Translationcited by 334
  7. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  8. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  9. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  10. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  11. 2019Evaluating Gender Bias in Machine Translationcited by 247
  12. 2019Massively Multilingual Neural Machine Translationcited by 246
  13. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  14. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  15. 2018Marian: Fast Neural Machine Translation in C++cited by 202
  16. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  17. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  18. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  19. 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
  20. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  21. 2017Unsupervised Neural Machine Translationcited by 148
  22. 2016Dual Learning for Machine Translationcited by 146
  23. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 143
  24. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  25. 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
  26. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  27. 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 124
  28. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  29. 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
  30. 2019Neural Machine Translation: A Review and Surveycited by 99
  31. 2019Multilingual Neural Machine Translation with Knowledge Distillationcited by 95
  32. 2020Low-resource Languages: A Review of Past Work and Future Challengescited by 90
  33. 2018Tensor2Tensor for Neural Machine Translationcited by 85
  34. 2018Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddingscited by 83
  35. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  36. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  37. 2019Tagged Back-Translationcited by 76
  38. 2020The Radicalization Risks of GPT-3 and Advanced Neural Language Modelscited by 76

50 most cited of 143 All 143 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 13; 2023: 21; 2024: 38; 2025: 18; 2026: 13.

108 papers.

Sort:

By area:

  1. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
  2. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  3. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 144
  4. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  5. 2021Survey of Low-Resource Machine Translationcited by 92
  6. 2022Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP Worldcited by 55
  7. 2023Unveiling Security, Privacy, and Ethical Concerns of ChatGPTcited by 53
  8. 2023Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimescited by 52
  9. 2021Multilingual training for Software Engineeringcited by 43
  10. 2024The Future of Large Language Model Pre-training is Federatedcited by 30
  11. 2024Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Researchcited by 26
  12. 2022Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?cited by 24
38 more
  1. 2023Beyond Static Datasets: A Deep Interaction Approach to LLM Evaluationcited by 24
  2. 2024Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmarkcited by 23
  3. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
  4. 2023JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processingcited by 18
  5. 2022USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translationcited by 15
  6. 2024Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corporacited by 13
  7. 2022Towards Better Chinese-centric Neural Machine Translation for Low-resource Languagescited by 11
  8. 2023Natural Language Processing in Ethiopian Languages: Current State, Challenges, and Opportunitiescited by 11
  9. 2023Large Human Language Models: A Need and the Challengescited by 11
  10. 2024Prompt Refinement with Image Pivot for Text-to-Image Generationcited by 11
  11. 2024FAIR Enough: How Can We Develop and Assess a FAIR-Compliant Dataset for Large Language Models' Training?cited by 9
  12. 2024Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitationcited by 9
  13. 2022Computational historical linguistics and language diversity in South Asiacited by 8
  14. 2024Brain-inspired Artificial Intelligence: A Comprehensive Reviewcited by 8
  15. 2023Neural Machine Translation for the Indigenous Languages of the Americas: An Introductioncited by 7
  16. 2023First Attempt at Building Parallel Corpora for Machine Translation of Northeast India's Very Low-Resource Languagescited by 7
  17. 2022A Theory of Unsupervised Translation Motivated by Understanding Animal Communicationcited by 6
  18. 2023An evaluation of Google Translate for Sanskrit to English translation via sentiment and semantic analysiscited by 6
  19. 2023Knowledge Relation Rank Enhanced Heterogeneous Learning Interaction Modeling for Neural Graph Forgetting Knowledge Tracingcited by 6
  20. 2023PuoBERTa: Training and evaluation of a curated language model for Setswanacited by 6
  21. 2024LLM-Assisted Rule Based Machine Translation for Low/No-Resource Languagescited by 6
  22. 2024Multistage Fine-tuning Strategies for Automatic Speech Recognition in Low-resource Languagescited by 6
  23. 2024Sinhala Transliteration: A Comparative Analysis Between Rule-based and Seq2Seq Approachescited by 6
  24. 2025Bhaasha, Bhasa, Zaban: A Survey for Low-Resourced Languages in South Asia -- Current Stage and Challengescited by 6
  25. 2021Training Cross-Lingual embeddings for Setswana and Sepedicited by 5
  26. 2023Low-Resourced Machine Translation for Senegalese Wolof Languagecited by 5
  27. 2023Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translationcited by 5
  28. 2024A Systematic Review of NLP for Ghanaian Languages: Datasets, Models, and a Research Roadmapcited by 5
  29. 2024Worldwide Federated Training of Language Modelscited by 5
  30. 2024On the Evaluation Practices in Multilingual NLP: Can Machine Translation Offer an Alternative to Human Translations?cited by 5
  31. 2024Rule-Based, Neural and LLM Back-Translation: Comparative Insights from a Variant of Ladincited by 5
  32. 2024Low-resource Machine Translation: what for? who for? An observational study on a dedicated Tetun language translation servicecited by 5
  33. 2025TLUE: A Tibetan Language Understanding Evaluation Benchmarkcited by 5
  34. 2022Improving Neural Machine Translation of Indigenous Languages with Multilingual Transfer Learningcited by 4
  35. 2023RoDia: A New Dataset for Romanian Dialect Identification from Speechcited by 4
  36. 2024Cross-Lingual Learning vs. Low-Resource Fine-Tuning: A Case Study with Fact-Checking in Turkishcited by 4
  37. 2024Representation Bias of Adolescents in AI: A Bilingual, Bicultural Studycited by 4
  38. 2021TinyDefectNet: Highly Compact Deep Neural Network Architecture for High-Throughput Manufacturing Visual Quality Inspectioncited by 3

50 most cited of 108 All 108 (JSON)