# Improving Neural Machine Translation of Indigenous Languages with Multilingual Transfer Learning

arXiv: https://arxiv.org/abs/2205.06993 · HTML: https://mlsanity.com/paper/2205.06993/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 14 May 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Marian: Fast Neural Machine Translation in C++](https://mlsanity.com/paper/1804.00344/) (2018)
- [Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1604.02201/) (2016)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation](https://mlsanity.com/paper/1708.09803/) (2017)
- [Trivial Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1809.00357/) (2018)
- [Challenges of language technologies for the indigenous languages of the Americas](https://mlsanity.com/paper/1806.04291/) (2018)
- [Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies](https://mlsanity.com/paper/1905.05475/) (2019)
- [IndT5: A Text-to-Text Transformer for 10 Indigenous Languages](https://mlsanity.com/paper/2104.07483/) (2021)
- [Lost in Translation: Analysis of Information Loss During Machine Translation Between Polysynthetic and Fusional Languages](https://mlsanity.com/paper/1807.00286/) (2018)

## Built on by (most cited first, 4 of 4)

- [NLP Progress in Indigenous Latin American Languages](https://mlsanity.com/paper/2404.05365/) (2024)
- [Harnessing the Power of Artificial Intelligence to Vitalize Endangered Indigenous Languages: Technologies and Experiences](https://mlsanity.com/paper/2407.12620/) (2024)
- [Leveraging the Cross-Domain & Cross-Linguistic Corpus for Low Resource NMT: A Case Study On Bhili-Hindi-English Parallel Corpus](https://mlsanity.com/paper/2511.00486/) (2025)
- [Building Efficient and Effective OpenQA Systems for Low-Resource Languages](https://mlsanity.com/paper/2401.03590/) (2024)

## Abstract (arXiv)

Machine translation (MT) involving Indigenous languages, including those possibly endangered, is challenging due to lack of sufficient parallel data. We describe an approach exploiting bilingual and multilingual pretrained MT models in a transfer learning setting to translate from Spanish to ten South American Indigenous languages. Our models set new SOTA on five out of the ten language pairs we consider, even doubling performance on one of these five pairs. Unlike previous SOTA that perform data augmentation to enlarge the train sets, we retain the low-resource setting to test the effectiveness of our models under such a constraint. In spite of the rarity of linguistic information available about the Indigenous languages, we offer a number of quantitative and qualitative analyses (e.g., as to morphology, tokenization, and orthography) to contextualize our results.