# Ensemble Self-Training for Unsupervised Machine Translation

arXiv: https://arxiv.org/abs/2603.17087 · HTML: https://mlsanity.com/paper/2603.17087/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 17 Mar 2026.

## Builds on (most cited first, 24 of 24)

- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020)
- [Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics](https://mlsanity.com/paper/2006.06264/) (2020)
- [Revisiting Self-Training for Neural Sequence Generation](https://mlsanity.com/paper/1909.13788/) (2019)
- [Unsupervised Neural Machine Translation](https://mlsanity.com/paper/1710.11041/) (2017)
- [A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'](https://mlsanity.com/paper/2410.12954/) (2024)
- [BLEU might be Guilty but References are not Innocent](https://mlsanity.com/paper/2004.06063/) (2020)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Neural Machine Translation: A Review and Survey](https://mlsanity.com/paper/1912.02047/) (2019)
- [The unreasonable effectiveness of few-shot learning for machine translation](https://mlsanity.com/paper/2302.01398/) (2023)
- [An Effective Approach to Unsupervised Machine Translation](https://mlsanity.com/paper/1902.01313/) (2019)
- [Eagle: Efficient Training-Free Router for Multi-LLM Inference](https://mlsanity.com/paper/2409.15518/) (2024)
- [When and Why is Unsupervised Neural Machine Translation Useless?](https://mlsanity.com/paper/2004.10581/) (2020)
- [Breaking the Ceiling of the LLM Community by Treating Token Generation as a Classification for Ensembling](https://mlsanity.com/paper/2406.12585/) (2024)
- [Unsupervised Neural Machine Translation with Generative Language Models Only](https://mlsanity.com/paper/2110.05448/) (2021)
- [A Multilingual View of Unsupervised Machine Translation](https://mlsanity.com/paper/2002.02955/) (2020)
- [BLEU, METEOR, BERTScore: Evaluation of Metrics Performance in Assessing Critical Translation Errors in Sentiment-oriented Text](https://mlsanity.com/paper/2109.14250/) (2021)
- [Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT](https://mlsanity.com/paper/2009.07610/) (2020)
- [Ensemble Learning for Large Language Models in Text and Code Generation: A Survey](https://mlsanity.com/paper/2503.13505/) (2025)

## Abstract (arXiv)

We present an ensemble-driven self-training framework for unsupervised neural machine translation (UNMT). Starting from a primary language pair, we train multiple UNMT models that share the same translation task but differ in an auxiliary language, inducing structured diversity across models. We then generate pseudo-translations for the primary pair using token-level ensemble decoding, averaging model predictions in both directions. These ensemble outputs are used as synthetic parallel data to further train each model, allowing the models to improve via shared supervision. At deployment time, we select a single model by validation performance, preserving single-model inference cost. Experiments show statistically significant improvements over single-model UNMT baselines, with mean gains of 1.7 chrF when translating from English and 0.67 chrF when translating into English.