# Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data

arXiv: https://arxiv.org/abs/2105.15071 · HTML: https://mlsanity.com/paper/2105.15071/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 47 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 30 of 36)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017)
- [CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB](https://mlsanity.com/paper/1911.04944/) (2019)
- [Unsupervised Neural Machine Translation](https://mlsanity.com/paper/1710.11041/) (2017)
- [Phrase-Based & Neural Unsupervised Machine Translation](https://mlsanity.com/paper/1804.07755/) (2018)
- [CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs](https://mlsanity.com/paper/1911.06154/) (2019)
- [When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?](https://mlsanity.com/paper/1804.06323/) (2018)
- [KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media](https://mlsanity.com/paper/2007.13184/) (2020)
- [Rapid Adaptation of Neural Machine Translation to New Languages](https://mlsanity.com/paper/1808.04189/) (2018)
- [The IIT Bombay English-Hindi Parallel Corpus](https://mlsanity.com/paper/1710.02855/) (2017)
- [TICO-19: the Translation Initiative for Covid-19](https://mlsanity.com/paper/2007.01788/) (2020)
- [Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations](https://mlsanity.com/paper/1906.01181/) (2019)
- [The Missing Ingredient in Zero-Shot Neural Machine Translation](https://mlsanity.com/paper/1903.07091/) (2019)
- [Cross-lingual Retrieval for Iterative Self-Supervised Training](https://mlsanity.com/paper/2006.09526/) (2020)
- [Generalized Data Augmentation for Low-Resource Translation](https://mlsanity.com/paper/1906.03785/) (2019)
- [Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2005.04816/) (2020)
- [DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder](https://mlsanity.com/paper/1805.12352/) (2018)
- [Automatic Language Identification in Texts: A Survey](https://mlsanity.com/paper/1804.08186/) (2018)
- [A Multilingual View of Unsupervised Machine Translation](https://mlsanity.com/paper/2002.02955/) (2020)
- [A Teacher-Student Framework for Zero-Resource Neural Machine Translation](https://mlsanity.com/paper/1705.00753/) (2017)

## Built on by (most cited first, 17 of 17)

- [A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation](https://mlsanity.com/paper/2205.02022/) (2022)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages](https://mlsanity.com/paper/2210.11621/) (2022)
- [Don't Go Far Off: An Empirical Study on Neural Poetry Translation](https://mlsanity.com/paper/2109.02972/) (2021)
- [Scaling Low-Resource MT via Synthetic Data Generation with LLMs](https://mlsanity.com/paper/2505.14423/) (2025)
- [Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu](https://mlsanity.com/paper/2502.11862/) (2025)
- [The Zeno's Paradox of `Low-Resource' Languages](https://mlsanity.com/paper/2410.20817/) (2024)
- [Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation](https://mlsanity.com/paper/2110.08547/) (2021)
- [Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions](https://mlsanity.com/paper/2211.01542/) (2022)
- [Letz Translate: Low-Resource Machine Translation for Luxembourgish](https://mlsanity.com/paper/2303.01347/) (2023)
- [Neural Machine Translation Data Generation and Augmentation using ChatGPT](https://mlsanity.com/paper/2307.05779/) (2023)
- [A Tulu Resource for Machine Translation](https://mlsanity.com/paper/2403.19142/) (2024)
- [Multilingual Bidirectional Unsupervised Translation Through Multilingual Finetuning and Back-Translation](https://mlsanity.com/paper/2209.02821/) (2022)
- [Extending Multilingual Machine Translation through Imitation Learning](https://mlsanity.com/paper/2311.08538/) (2023)
- [Exploiting Multilingualism in Low-resource Neural Machine Translation via Adversarial Learning](https://mlsanity.com/paper/2303.18011/) (2023)
- [Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications](https://mlsanity.com/paper/2109.08627/) (2021)
- [Deterministic Reversible Data Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2406.02517/) (2024)

## Abstract (arXiv)

The scarcity of parallel data is a major obstacle for training high-quality machine translation systems for low-resource languages. Fortunately, some low-resource languages are linguistically related or similar to high-resource languages; these related languages may share many lexical or syntactic structures. In this work, we exploit this linguistic overlap to facilitate translating to and from a low-resource language with only monolingual data, in addition to any parallel data in the related high-resource language. Our method, NMT-Adapt, combines denoising autoencoding, back-translation and adversarial objectives to utilize monolingual data for low-resource adaptation. We experiment on 7 languages from three different language families and show that our technique significantly improves translation into low-resource language compared to other translation baselines.