mlsanityData to 7 Oct 2026

A Survey on Low-Resource Neural Machine Translation

arXiv 2107.04239 ↗First posted 9 Jul 2021cs.CL cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data. Thus, a lot of research has been conducted for neural machine translation (NMT) with very limited parallel data, i.e., the low-resource setting. In this paper, we provide a survey for low-resource NMT and classify related works into three categories according to the auxiliary data they used: (1) exploiting monolingual data of source and/or target languages, (2) exploiting data from auxiliary languages, and (3) exploiting multi-modal data. We hope that our survey can help researchers to better understand this field and inspire them to design better algorithms, and help industry practitioners to choose appropriate algorithms for their applications.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 101 of its 161 references that are in the map. First posted on arXiv 9 Jul 2021.

Builds on

101 of 161 references.

References · 101

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  9. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  10. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  12. 2016Enriching Word Vectors with Subword Informationcited by 1,578
38 more
  1. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  2. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  3. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  4. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  5. 2017Word Translation Without Parallel Datacited by 480
  6. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  7. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  8. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  9. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  10. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  11. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  12. 2019Massively Multilingual Neural Machine Translationcited by 246
  13. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  14. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  15. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  16. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  17. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  18. 2016Exploring the Limits of Language Modelingcited by 172
  19. 2017Unsupervised Neural Machine Translationcited by 148
  20. 2016Dual Learning for Machine Translationcited by 146
  21. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 145
  22. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 143
  23. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  24. 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
  25. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  26. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  27. 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
  28. 2019Multilingual Neural Machine Translation with Knowledge Distillationcited by 95
  29. 2015Multi-task Sequence to Sequence Learningcited by 94
  30. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  31. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  32. 2019Tagged Back-Translationcited by 76
  33. 2020Balancing Training for Multilingual Neural Machine Translationcited by 71
  34. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  35. 2018A Survey of Domain Adaptation for Neural Machine Translationcited by 64
  36. 2019The Missing Ingredient in Zero-Shot Neural Machine Translationcited by 59
  37. 2017Transfer Learning across Low-Resource, Related Languages for Neural Machine Translationcited by 58
  38. 2016A Convolutional Encoder Model for Neural Machine Translationcited by 54

50 most cited of 101 All 101 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 6; 2024: 3; 2025: 2; 2026: 2.

16 papers.

Sort:

  1. 2021Survey of Low-Resource Machine Translationcited by 92
  2. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  3. 2022Towards Better Chinese-centric Neural Machine Translation for Low-resource Languagescited by 11
  4. 2023Assessing Translation capabilities of Large Language Models involving English and Indian Languagescited by 8
  5. 2025Overcoming Data Scarcity in Generative Language Modelling for Low-Resource Languages: A Systematic Reviewcited by 8
  6. 2023Exploring Methods for Building Dialects-Mandarin Code-Mixing Corpora: A Case Study in Taiwanese Hokkiencited by 7
  7. 2023Neural Machine Translation for the Indigenous Languages of the Americas: An Introductioncited by 7
  8. 2023Extract and Attend: Improving Entity Translation in Neural Machine Translationcited by 2
  9. 2024Low-Resource Machine Translation through the Lens of Personalized Federated Learningcited by 1
  10. 2024Dialectal and Low-Resource Machine Translation for Aromaniancited by 1
  11. 2024From Priest to Doctor: Domain Adaptation for Low-Resource Neural Machine Translationcited by 1
  12. 2022Blur the Linguistic Boundary: Interpreting Chinese Buddhist Sutra in English via Neural Machine Translationcited by 0
4 more
  1. 2023Unraveling Key Factors of Knowledge Distillationcited by 0
  2. 2025VNJPTranslate: A comprehensive pipeline for Vietnamese-Japanese translationcited by 0
  3. 2026Multilingual Reasoning Cascades Need More Contextcited by 0
  4. 2026Systematic Analysis of Large Language Models and Transformer-Based Machine Translation for English-Tamil and Tamil-English Across Diverse Datasetscited by 0