mlsanityData to 7 Oct 2026

Transfer Learning for Low-Resource Neural Machine Translation

arXiv 1604.02201 ↗First posted 8 Apr 2016cs.CLcited by 188

Not on the map. Why?

Abstract (arXiv)

The encoder-decoder framework for neural machine translation (NMT) has been shown effective in large data scenarios, but is much less effective for low-resource languages. We present a transfer learning method that significantly improves Bleu scores across a range of low-resource languages. Our key idea is to first train a high-resource language pair (the parent model), then transfer some of the learned parameters to the low-resource pair (the child model) to initialize and constrain training. Using our transfer learning method we improve baseline NMT models by an average of 5.6 Bleu on four low-resource language pairs. Ensembling and unknown word replacement add another 2 Bleu which brings the NMT performance on low-resource machine translation close to a strong syntax based machine translation (SBMT) system, exceeding its performance on one language pair. Additionally, using the transfer learning model for re-scoring, we can improve the SBMT system by an average of 1.3 Bleu, improving the state-of-the-art on low-resource machine translation.

Key facts. Cited by 188 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 59; 2022: 40; 2023: 31; 2024: 33; 2025: 12; 2026: 12.

188 papers.

Sort:

By area:

  1. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  2. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  3. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  4. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
  5. 2021Survey of Low-Resource Machine Translationcited by 92
  6. 2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
  7. 2021Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translationcited by 74
  8. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 55
  9. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  10. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  11. 2021The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translationcited by 46
  12. 2021The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translationcited by 38
38 more
  1. 2021Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitutioncited by 31
  2. 2021Many-to-English Machine Translation Tools, Data, and Pretrained Modelscited by 31
  3. 2021Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translationcited by 29
  4. 2021Data Augmentation for Sign Language Gloss Translationcited by 28
  5. 2022English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddingscited by 26
  6. 2023Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learningcited by 26
  7. 2024Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Researchcited by 26
  8. 2021AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languagescited by 24
  9. 2021Importance-based Neuron Allocation for Multilingual Neural Machine Translationcited by 23
  10. 2022Listening to Affected Communities to Define Extreme Speech: Dataset and Experimentscited by 21
  11. 2024ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Modelscited by 20
  12. 2019Learning Norms from Stories: A Prior for Value Aligned Agentscited by 19
  13. 2022Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learningcited by 19
  14. 2023Aligning Neural Machine Translation Models: Human Feedback in Training and Inferencecited by 19
  15. 2021How Good Is NLP? A Sober Look at NLP Tasks through the Lens of Social Impactcited by 18
  16. 2021Distributionally Robust Multilingual Machine Translationcited by 18
  17. 2022Detecting Individual Decision-Making Style: Exploring Behavioral Stylometry in Chesscited by 18
  18. 2024Abstractive Text Summarization: State of the Art, Challenges, and Improvementscited by 17
  19. 2024Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoningcited by 17
  20. 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 16
  21. 2021A Survey on Low-Resource Neural Machine Translationcited by 16
  22. 2021Teacher Model Fingerprinting Attacks Against Transfer Learningcited by 15
  23. 2023Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lenscited by 15
  24. 2023TADA: Task-Agnostic Dialect Adapters for Englishcited by 15
  25. 2021Training Value-Aligned Reinforcement Learning Agents Using a Normative Priorcited by 14
  26. 2022Language-Family Adapters for Low-Resource Multilingual Neural Machine Translationcited by 14
  27. 2025Scaling Low-Resource MT via Synthetic Data Generation with LLMscited by 14
  28. 2024Low-Resource Vision Challenges for Foundation Modelscited by 13
  29. 2024Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approachcited by 13
  30. 2021Adapting Monolingual Models: Data can be Scarce when Language Similarity is Highcited by 12
  31. 2025Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchucited by 12
  32. 2022Data Selection Curriculum for Neural Machine Translationcited by 11
  33. 2022MMTAfrica: Multilingual Machine Translation for African Languagescited by 11
  34. 2022Towards Better Chinese-centric Neural Machine Translation for Low-resource Languagescited by 11
  35. 2022On the Generalization for Transfer Learning: An Information-Theoretic Analysiscited by 11
  36. 2021IndT5: A Text-to-Text Transformer for 10 Indigenous Languagescited by 10
  37. 2022Learning to Parallelize in a Shared-Memory Environment with Transformerscited by 10
  38. 2022SQuId: Measuring Speech Naturalness in Many Languagescited by 10

50 most cited of 188 All 188 (JSON)