mlsanityData to 14 Oct 2026

Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Machine Translation

arXiv 2104.05848 ↗First posted 12 Apr 2021cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

We translate a closed text that is known in advance into a severely low resource language by leveraging massive source parallelism. In other words, given a text in 124 source languages, we translate it into a severely low resource language using only ~1,000 lines of low resource data without any external help. Firstly, we propose a systematic method to rank and choose source languages that are close to the low resource language. We call the linguistic definition of language family Family of Origin (FAMO), and we call the empirical definition of higher-ranked languages using our metrics Family of Choice (FAMC). Secondly, we build an Iteratively Pretrained Multilingual Order-preserving Lexiconized Transformer (IPML) to train on ~1,000 lines (~3.5%) of low resource data. To translate named entities correctly, we build a massive lexicon table for 2,939 Bible named entities in 124 source languages, and include many that occur once and covers more than 66 severely low resource languages. Moreover, we also build a novel method of combining translations from different source languages into one. Using English as a hypothetical low resource language, we get a +23.9 BLEU increase over a multilingual baseline, and a +10.3 BLEU increase over our asymmetric baseline in the Bible dataset. We get a 42.8 BLEU score for Portuguese-English translation on the medical EMEA dataset. We also have good results for a real severely low resource Mayan language, Eastern Pokomchi.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 81 references that are in the map. First posted on arXiv 12 Apr 2021.

Builds on

33 of 81 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  4. 2014Neural Turing Machinescited by 644
  5. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  6. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  7. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  8. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  9. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  10. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 144
  11. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 126
  12. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
21 more
  1. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  2. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  3. 2020Zero-Shot Cross-Lingual Transfer with Meta Learningcited by 66
  4. 2020Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Informationcited by 64
  5. 2014Addressing the Rare Word Problem in Neural Machine Translationcited by 50
  6. 2020Language Model Prior for Low-Resource Neural Machine Translationcited by 37
  7. 2020Complete Multilingual Neural Machine Translationcited by 30
  8. 2015Multilingual Language Processing From Bytescited by 26
  9. 2020Dynamic Data Selection and Weighting for Iterative Back-Translationcited by 25
  10. 2020Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representationscited by 21
  11. 2020On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatmentcited by 21
  12. 2020Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMTcited by 18
  13. 2020Iterative Domain-Repaired Back-Translationcited by 13
  14. 2018What Level of Quality can Neural Machine Translation Attain on Literary Text?cited by 11
  15. 2018Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translationcited by 10
  16. 2017Neural machine translation for low-resource languagescited by 8
  17. 2020A Multilingual Neural Machine Translation Model for Biomedical Datacited by 7
  18. 2018Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translationcited by 6
  19. 2020Translating Similar Languages: Role of Mutual Intelligibility in Multilingual Transformerscited by 6
  20. 2018Paraphrases as Foreign Languages in Multilingual Neural Machine Translationcited by 5
  21. 2016Weakly supervised spoken term discovery using cross-lingual side informationcited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 1; 2024: 1.

4 papers.

  1. 2022Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?cited by 24
  2. 2021Active Learning for Massively Parallel Translation of Constrained Text into Low Resource Languagescited by 4
  3. 2024Exploiting Domain-Specific Parallel Data on Multilingual Language Models for Low-resource Language Translationcited by 2
  4. 2023Train Global, Tailor Local: Minimalist Multilingual Translation into Endangered Languagescited by 0