# Training Multilingual Machine Translation by Alternately Freezing Language-Specific Encoders-Decoders

arXiv: https://arxiv.org/abs/2006.01594 · HTML: https://mlsanity.com/paper/2006.01594/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2020.

## Built on by (most cited first, 8 of 8)

- [Contrastive Learning for Many-to-many Multilingual Neural Machine Translation](https://mlsanity.com/paper/2105.09501/) (2021)
- [T-Modules: Translation Modules for Zero-Shot Cross-Modal Machine Translation](https://mlsanity.com/paper/2205.12216/) (2022)
- [Continual Learning in Multilingual NMT via Language-Specific Embeddings](https://mlsanity.com/paper/2110.10478/) (2021)
- [Back-translation for Large-Scale Multilingual Machine Translation](https://mlsanity.com/paper/2109.08712/) (2021)
- [Code-switched inspired losses for generic spoken dialog representations](https://mlsanity.com/paper/2108.12465/) (2021)
- [Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders](https://mlsanity.com/paper/2102.06578/) (2021)
- [Forklift: An Extensible Neural Lifter](https://mlsanity.com/paper/2404.16041/) (2024)
- [Contrastive Token Learning with Similarity Decay for Repetition Suppression in Machine Translation](https://mlsanity.com/paper/2409.19877/) (2024)

## Abstract (arXiv)

We propose a modular architecture of language-specific encoder-decoders that constitutes a multilingual machine translation system that can be incrementally extended to new languages without the need for retraining the existing system when adding new languages. Differently from previous works, we simultaneously train N languages in all translation directions by alternately freezing encoder or decoder modules, which indirectly forces the system to train in a common intermediate representation for all languages. Experimental results from multilingual machine translation show that we can successfully train this modular architecture improving on the initial languages while falling slightly behind when adding new languages or doing zero-shot translation. Additional comparison of the quality of sentence representation in the task of natural language inference shows that the alternately freezing training is also beneficial in this direction.