# Back-translation for Large-Scale Multilingual Machine Translation

arXiv: https://arxiv.org/abs/2109.08712 · HTML: https://mlsanity.com/paper/2109.08712/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 20 references that are in the map.
- First posted on arXiv 17 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation](https://mlsanity.com/paper/2004.11867/) (2020)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [Analyzing Uncertainty in Neural Machine Translation](https://mlsanity.com/paper/1803.00047/) (2018)
- [Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies](https://mlsanity.com/paper/1905.05475/) (2019)
- [Generalizing Back-Translation in Neural Machine Translation](https://mlsanity.com/paper/1906.07286/) (2019)
- [Enabling Multi-Source Neural Machine Translation By Concatenating Source Sentences In Multiple Languages](https://mlsanity.com/paper/1702.06135/) (2017)
- [Multilingual Multi-Domain Adaptation Approaches for Neural Machine Translation](https://mlsanity.com/paper/1906.07978/) (2019)
- [Training Multilingual Machine Translation by Alternately Freezing Language-Specific Encoders-Decoders](https://mlsanity.com/paper/2006.01594/) (2020)
- [Improving a Multi-Source Neural Machine Translation Model with Corpus Extension for Low-Resource Languages](https://mlsanity.com/paper/1709.08898/) (2017)

## Built on by (most cited first, 11 of 11)

- [Best Practices and Lessons Learned on Synthetic Data](https://mlsanity.com/paper/2404.07503/) (2024)
- [Enhancing LLM Safety Through a Theoretical Minimax Game Lens](https://mlsanity.com/paper/2502.05163/) (2025)
- [Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens](https://mlsanity.com/paper/2305.11550/) (2023)
- [Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation](https://mlsanity.com/paper/2305.12786/) (2023)
- [Backdoor Attack on Multilingual Machine Translation](https://mlsanity.com/paper/2404.02393/) (2024)
- [Learning an Artificial Language for Knowledge-Sharing in Multilingual Translation](https://mlsanity.com/paper/2211.01292/) (2022)
- [Ask Language Model to Clean Your Noisy Translation Data](https://mlsanity.com/paper/2310.13469/) (2023)
- [IKUN for WMT24 General MT Task: LLMs Are here for Multilingual Machine Translation](https://mlsanity.com/paper/2408.11512/) (2024)
- [Low-Resource Machine Translation through the Lens of Personalized Federated Learning](https://mlsanity.com/paper/2406.12564/) (2024)
- [The VolcTrans System for WMT22 Multilingual Machine Translation Task](https://mlsanity.com/paper/2210.11599/) (2022)
- [University of Cape Town's WMT22 System: Multilingual Machine Translation for Southern African Languages](https://mlsanity.com/paper/2210.11757/) (2022)

## Abstract (arXiv)

This paper illustrates our approach to the shared task on large-scale multilingual machine translation in the sixth conference on machine translation (WMT-21). This work aims to build a single multilingual translation system with a hypothesis that a universal cross-language representation leads to better multilingual translation performance. We extend the exploration of different back-translation methods from bilingual translation to multilingual translation. Better performance is obtained by the constrained sampling method, which is different from the finding of the bilingual translation. Besides, we also explore the effect of vocabularies and the amount of synthetic data. Surprisingly, the smaller size of vocabularies perform better, and the extensive monolingual English data offers a modest improvement. We submitted to both the small tasks and achieved the second place.