mlsanityData to 7 Oct 2026

Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation

arXiv 1708.09803 ↗First posted 31 Aug 2017cs.CLcited by 58

Not on the map. Why?

Abstract (arXiv)

We present a simple method to improve neural translation of a low-resource language pair using parallel data from a related, also low-resource, language pair. The method is based on the transfer method of Zoph et al., but whereas their method ignores any source vocabulary overlap, ours exploits it. First, we split words using Byte Pair Encoding (BPE) to increase vocabulary overlap. Then, we train a model on the first language pair and transfer its parameters, including its source word embeddings, to another model and continue training on the second language pair. Our experiments show that transfer learning helps word-based translation only slightly, but when used on top of a much stronger BPE baseline, it yields larger improvements of up to 4.3 BLEU.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Aug 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 11; 2023: 7; 2024: 5; 2025: 2; 2026: 4.

58 papers.

Sort:

  1. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 133
  2. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
  3. 2021Survey of Low-Resource Machine Translationcited by 92
  4. 2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
  5. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  6. 2021Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translationcited by 74
  7. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 65
  8. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 55
  9. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  10. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  11. 2021IndicBART: A Pre-trained Model for Indic Natural Language Generationcited by 39
  12. 2023Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learningcited by 26
38 more
  1. 2021Predicting the Performance of Multilingual NLP Modelscited by 24
  2. 2022Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learningcited by 19
  3. 2022Aksharantar: Open Indic-language Transliteration datasets and models for the Next Billion Userscited by 19
  4. 2021An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformerscited by 18
  5. 2021A Survey on Low-Resource Neural Machine Translationcited by 16
  6. 2021Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Banditscited by 16
  7. 2021Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generationcited by 14
  8. 2021Subword Mapping and Anchoring across Languagescited by 12
  9. 2022MicroBERT: Effective Training of Low-resource Monolingual BERTs through Parameter Reduction and Multitask Learningcited by 12
  10. 2021IndT5: A Text-to-Text Transformer for 10 Indigenous Languagescited by 10
  11. 2021Low-Resource Neural Machine Translation for Southern African Languagescited by 7
  12. 2021Prose2Poem: The Blessing of Transformers in Translating Prose to Persian Poetrycited by 7
  13. 2023Neural Machine Translation for the Indigenous Languages of the Americas: An Introductioncited by 7
  14. 2021Improving Arabic Diacritization by Learning to Diacritize and Translatecited by 6
  15. 2022Improving Multilingual Neural Machine Translation System for Indic Languagescited by 6
  16. 2022BERT in Plutarch's Shadowscited by 6
  17. 2024Recent Advancements and Challenges of Turkic Central Asian Language Processingcited by 6
  18. 2021Translating the Unseen? Yoruba-English MT in Low-Resource, Morphologically-Unmarked Settingscited by 5
  19. 2021"Wikily" Supervised Neural Translation Tailored to Cross-Lingual Taskscited by 5
  20. 2022Geographical Distance Is The New Hyperparameter: A Case Study Of Finding The Optimal Pre-trained Language For English-isiZulu Machine Translationcited by 5
  21. 2021Token-wise Curriculum Learning for Neural Machine Translationcited by 4
  22. 2022DTW at Qur'an QA 2022: Utilising Transfer Learning with Transformers for Question Answering in a Low-resource Domaincited by 4
  23. 2022Improving Neural Machine Translation of Indigenous Languages with Multilingual Transfer Learningcited by 4
  24. 2021Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languagescited by 3
  25. 2021Cross-lingual Transferring of Pre-trained Contextualized Language Modelscited by 3
  26. 2021HintedBT: Augmenting Back-Translation with Quality and Transliteration Hintscited by 3
  27. 2022Sub-Word Alignment Is Still Useful: A Vest-Pocket Method for Enhancing Low-Resource Machine Translationcited by 3
  28. 2022OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrievalcited by 3
  29. 2024UMBCLU at SemEval-2024 Task 1A and 1C: Semantic Textual Relatedness with and without machine translationcited by 3
  30. 2024Tagengo: A Multilingual Chat Datasetcited by 3
  31. 2021Semi-supervised transfer learning for language expansion of end-to-end speech recognition models to low-resource languagescited by 2
  32. 2023Izindaba-Tindzaba: Machine learning news categorisation for Long and Short Text for isiZulu and Siswaticited by 2
  33. 2024Could We Have Had Better Multilingual LLMs If English Was Not the Central Language?cited by 2
  34. 2025Zero-Shot Performance Prediction for Probabilistic Scaling Lawscited by 2
  35. 2021Self-Learning for Zero Shot Neural Machine Translationcited by 1
  36. 2021Contextual Text Embeddings for Twicited by 1
  37. 2021Machine Translation of Low-Resource Indo-European Languagescited by 1
  38. 2023Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languagescited by 1

50 most cited of 58 All 58 (JSON)