# Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders

arXiv: https://arxiv.org/abs/2004.06575 · HTML: https://mlsanity.com/paper/2004.06575/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2020.

## Built on by (most cited first, 23 of 23)

- [Gender Bias in Machine Translation](https://mlsanity.com/paper/2104.06001/) (2021)
- [Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages](https://mlsanity.com/paper/2205.12654/) (2022)
- [Importance-based Neuron Allocation for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2107.06569/) (2021)
- [T-Modules: Translation Modules for Zero-Shot Cross-Modal Machine Translation](https://mlsanity.com/paper/2205.12216/) (2022)
- [Pushing the Limits of Zero-shot End-to-End Speech Translation](https://mlsanity.com/paper/2402.10422/) (2024)
- [Continual Learning in Multilingual NMT via Language-Specific Embeddings](https://mlsanity.com/paper/2110.10478/) (2021)
- [Efficient Inference for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2109.06679/) (2021)
- [Adapting to Non-Centered Languages for Zero-shot Multilingual Translation](https://mlsanity.com/paper/2209.04138/) (2022)
- [Parameter Differentiation based Multilingual Neural Machine Translation](https://mlsanity.com/paper/2112.13619/) (2021)
- [HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2207.04906/) (2022)
- [Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders](https://mlsanity.com/paper/2102.06578/) (2021)
- [Condensing Multilingual Knowledge with Lightweight Language-Specific Modules](https://mlsanity.com/paper/2305.13993/) (2023)
- [Revisiting Language Encoding in Learning Multilingual Representations](https://mlsanity.com/paper/2102.08357/) (2021)
- [An Empirical Investigation of Multi-bridge Multilingual NMT models](https://mlsanity.com/paper/2110.07304/) (2021)
- [DATScore: Evaluating Translation with Data Augmented Translations](https://mlsanity.com/paper/2210.06576/) (2022)
- [Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages](https://mlsanity.com/paper/2302.03528/) (2023)
- [MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki](https://mlsanity.com/paper/2403.07544/) (2024)
- [More Parameters? No Thanks!](https://mlsanity.com/paper/2107.09622/) (2021)
- [I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures](https://mlsanity.com/paper/2404.17918/) (2024)
- [Exploring Intrinsic Language-specific Subspaces in Fine-tuning Multilingual Neural Machine Translation](https://mlsanity.com/paper/2409.05224/) (2024)
- [Cross-Lingual Representation Alignment Through Contrastive Image-Caption Tuning](https://mlsanity.com/paper/2505.13628/) (2025)
- [The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translation](https://mlsanity.com/paper/2605.04196/) (2026)
- [The Interlingua Hypothesis: LLMs Translate via a Latent Task-agnostic Feature Space](https://mlsanity.com/paper/2609.00515/) (2026)

## Abstract (arXiv)

State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. In this paper, we propose an alternative approach that is based on language-specific encoder-decoders, and can thus be more easily extended to new languages by learning their corresponding modules. So as to encourage a common interlingua representation, we simultaneously train the N initial languages. Our experiments show that the proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average, and when adding new languages, without the need to retrain the rest of the modules. All in all, our work closes the gap between shared and language-specific encoder-decoders, advancing toward modular multilingual machine translation systems that can be flexibly extended in lifelong learning settings.