# Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios

arXiv: https://arxiv.org/abs/2004.04507 · HTML: https://mlsanity.com/paper/2004.04507/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2020.

## Built on by (most cited first, 6 of 6)

- [Reinforced Self-Training (ReST) for Language Modeling](https://mlsanity.com/paper/2308.08998/) (2023)
- [SIKeD: Self-guided Iterative Knowledge Distillation for mathematical reasoning](https://mlsanity.com/paper/2410.18574/) (2024)
- [Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2203.08394/) (2022)
- [SMART: Self-learning Meta-strategy Agent for Reasoning Tasks](https://mlsanity.com/paper/2410.16128/) (2024)
- [Multi-Domain Targeted Sentiment Analysis](https://mlsanity.com/paper/2205.03804/) (2022)
- [CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2210.05146/) (2022)

## Abstract (arXiv)

Unsupervised neural machine translation (UNMT) that relies solely on massive monolingual corpora has achieved remarkable results in several translation tasks. However, in real-world scenarios, massive monolingual corpora do not exist for some extremely low-resource languages such as Estonian, and UNMT systems usually perform poorly when there is not adequate training corpus for one language. In this paper, we first define and analyze the unbalanced training data scenario for UNMT. Based on this scenario, we propose UNMT self-training mechanisms to train a robust UNMT system and improve its performance in this case. Experimental results on several language pairs show that the proposed methods substantially outperform conventional UNMT systems.