# Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages

arXiv: https://arxiv.org/abs/2009.11201 · HTML: https://mlsanity.com/paper/2009.11201/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2020.

## Built on by (most cited first, 22 of 22)

- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [The unreasonable effectiveness of few-shot learning for machine translation](https://mlsanity.com/paper/2302.01398/) (2023)
- [Building Machine Translation Systems for the Next Thousand Languages](https://mlsanity.com/paper/2205.03983/) (2022)
- [Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts](https://mlsanity.com/paper/2306.11372/) (2023)
- [LLM Augmented LLMs: Expanding Capabilities through Composition](https://mlsanity.com/paper/2401.02412/) (2024)
- [Using natural language prompts for machine translation](https://mlsanity.com/paper/2202.11822/) (2022)
- [Multilingual Unsupervised Neural Machine Translation with Denoising Adapters](https://mlsanity.com/paper/2110.10472/) (2021)
- [Machine Translation into Low-resource Language Varieties](https://mlsanity.com/paper/2106.06797/) (2021)
- [Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning](https://mlsanity.com/paper/2201.03110/) (2022)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data](https://mlsanity.com/paper/2105.15071/) (2021)
- [SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages](https://mlsanity.com/paper/2210.11621/) (2022)
- [Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation](https://mlsanity.com/paper/2110.08547/) (2021)
- [Towards Universality in Multilingual Text Rewriting](https://mlsanity.com/paper/2107.14749/) (2021)
- [Unsupervised Machine Translation On Dravidian Languages](https://mlsanity.com/paper/2103.15877/) (2021)
- [Multilingual Bidirectional Unsupervised Translation Through Multilingual Finetuning and Back-Translation](https://mlsanity.com/paper/2209.02821/) (2022)
- [Cost-Effective Training in Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2201.05700/) (2022)
- [Refining Low-Resource Unsupervised Translation by Language Disentanglement of Multilingual Model](https://mlsanity.com/paper/2205.15544/) (2022)
- [Unsupervised Translation of German--Lower Sorbian: Exploring Training and Novel Transfer Methods on a Low-Resource Language](https://mlsanity.com/paper/2109.12012/) (2021)
- [Boosting Unsupervised Machine Translation with Pseudo-Parallel Data](https://mlsanity.com/paper/2310.14262/) (2023)
- [An Empirical study of Unsupervised Neural Machine Translation: analyzing NMT output, model's behavior and sentences' contribution](https://mlsanity.com/paper/2312.12588/) (2023)
- [Unsupervised Cross-Lingual Part-of-Speech Tagging with Monolingual Corpora Only](https://mlsanity.com/paper/2602.09366/) (2026)

## Abstract (arXiv)

Unsupervised translation has reached impressive performance on resource-rich language pairs such as English-French and English-German. However, early studies have shown that in more realistic settings involving low-resource, rare languages, unsupervised translation performs poorly, achieving less than 3.0 BLEU. In this work, we show that multilinguality is critical to making unsupervised systems practical for low-resource settings. In particular, we present a single model for 5 low-resource languages (Gujarati, Kazakh, Nepali, Sinhala, and Turkish) to and from English directions, which leverages monolingual and auxiliary parallel data from other high-resource language pairs via a three-stage training scheme. We outperform all current state-of-the-art unsupervised baselines for these languages, achieving gains of up to 14.4 BLEU. Additionally, we outperform a large collection of supervised WMT submissions for various language pairs as well as match the performance of the current state-of-the-art supervised model for Nepali-English. We conduct a series of ablation studies to establish the robustness of our model under different degrees of data quality, as well as to analyze the factors which led to the superior performance of the proposed approach over traditional unsupervised models.