# Complete Multilingual Neural Machine Translation

arXiv: https://arxiv.org/abs/2010.10239 · HTML: https://mlsanity.com/paper/2010.10239/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2020.

## Built on by (most cited first, 30 of 30)

- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [MADLAD-400: A Multilingual And Document-Level Large Audited Dataset](https://mlsanity.com/paper/2309.04662/) (2023)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Causes and Cures for Interference in Multilingual Translation](https://mlsanity.com/paper/2212.07530/) (2022)
- [Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters](https://mlsanity.com/paper/2110.09574/) (2021)
- [Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training](https://mlsanity.com/paper/2109.02284/) (2021)
- [Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits](https://mlsanity.com/paper/2110.06997/) (2021)
- [Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens](https://mlsanity.com/paper/2305.11550/) (2023)
- [Learning Language-Specific Layers for Multilingual Machine Translation](https://mlsanity.com/paper/2305.02665/) (2023)
- [Continual Learning in Multilingual NMT via Language-Specific Embeddings](https://mlsanity.com/paper/2110.10478/) (2021)
- [Efficient Inference for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2109.06679/) (2021)
- [Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding](https://mlsanity.com/paper/2311.02851/) (2023)
- [Vega-MT: The JD Explore Academy Translation System for WMT22](https://mlsanity.com/paper/2209.09444/) (2022)
- [Evaluating Multiway Multilingual NMT in the Turkic Languages](https://mlsanity.com/paper/2109.06262/) (2021)
- [Scaling Sign Language Translation](https://mlsanity.com/paper/2407.11855/) (2024)
- [Adapting to Non-Centered Languages for Zero-shot Multilingual Translation](https://mlsanity.com/paper/2209.04138/) (2022)
- [Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Machine Translation](https://mlsanity.com/paper/2104.05848/) (2021)
- [Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations](https://mlsanity.com/paper/2206.14982/) (2022)
- [Learning an Artificial Language for Knowledge-Sharing in Multilingual Translation](https://mlsanity.com/paper/2211.01292/) (2022)
- [Data-adaptive Transfer Learning for Translation: A Case Study in Haitian and Jamaican](https://mlsanity.com/paper/2209.06295/) (2022)
- [An Empirical Investigation of Multi-bridge Multilingual NMT models](https://mlsanity.com/paper/2110.07304/) (2021)
- [EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation](https://mlsanity.com/paper/2203.02180/) (2022)
- [Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation](https://mlsanity.com/paper/2208.05852/) (2022)
- [Informative Language Representation Learning for Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/2209.01530/) (2022)
- [OpusCleaner and OpusTrainer, open source toolkits for training Machine Translation and Large language models](https://mlsanity.com/paper/2311.14838/) (2023)
- [Analyzing Architectures for Neural Machine Translation Using Low Computational Resources](https://mlsanity.com/paper/2111.03813/) (2021)
- [Direct Neural Machine Translation with Task-level Mixture of Experts models](https://mlsanity.com/paper/2310.12236/) (2023)
- [Massively Multilingual Text Translation For Low-Resource Languages](https://mlsanity.com/paper/2401.16582/) (2024)

## Abstract (arXiv)

Multilingual Neural Machine Translation (MNMT) models are commonly trained on a joint set of bilingual corpora which is acutely English-centric (i.e. English either as the source or target language). While direct data between two languages that are non-English is explicitly available at times, its use is not common. In this paper, we first take a step back and look at the commonly used bilingual corpora (WMT), and resurface the existence and importance of implicit structure that existed in it: multi-way alignment across examples (the same sentence in more than two languages). We set out to study the use of multi-way aligned examples to enrich the original English-centric parallel corpora. We reintroduce this direct parallel data from multi-way aligned corpora between all source and target languages. By doing so, the English-centric graph expands into a complete graph, every language pair being connected. We call MNMT with such connectivity pattern complete Multilingual Neural Machine Translation (cMNMT) and demonstrate its utility and efficacy with a series of experiments and analysis. In combination with a novel training data sampling strategy that is conditioned on the target language only, cMNMT yields competitive translation quality for all language pairs. We further study the size effect of multi-way aligned data, its transfer learning capabilities and how it eases adding a new language in MNMT. Finally, we stress test cMNMT at scale and demonstrate that we can train a cMNMT model with up to 111*112=12,432 language pairs that provides competitive translation quality for all language pairs.