# From Bilingual to Multilingual Neural Machine Translation by Incremental Training

arXiv: https://arxiv.org/abs/1907.00735 · HTML: https://mlsanity.com/paper/1907.00735/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2019.

## Built on by (most cited first, 14 of 14)

- [Towards Lifelong Learning of Large Language Models: A Survey](https://mlsanity.com/paper/2406.06391/) (2024)
- [Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution](https://mlsanity.com/paper/2103.06799/) (2021)
- [Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens](https://mlsanity.com/paper/2305.11550/) (2023)
- [Continual Learning in Multilingual NMT via Language-Specific Embeddings](https://mlsanity.com/paper/2110.10478/) (2021)
- [How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation](https://mlsanity.com/paper/2105.13782/) (2021)
- [Life-long Learning for Multilingual Neural Machine Translation with Knowledge Distillation](https://mlsanity.com/paper/2212.02800/) (2022)
- [Self-generated Replay Memories for Continual Neural Machine Translation](https://mlsanity.com/paper/2403.13130/) (2024)
- [Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders](https://mlsanity.com/paper/2102.06578/) (2021)
- [Multilingual k-Nearest-Neighbor Machine Translation](https://mlsanity.com/paper/2310.14644/) (2023)
- [Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages](https://mlsanity.com/paper/2302.03528/) (2023)
- [Forklift: An Extensible Neural Lifter](https://mlsanity.com/paper/2404.16041/) (2024)
- [Unsupervised Transfer Learning in Multilingual Neural Machine Translation with Cross-Lingual Word Embeddings](https://mlsanity.com/paper/2103.06689/) (2021)
- [How effective is Multi-source pivoting for Translation of Low Resource Indian Languages?](https://mlsanity.com/paper/2406.13332/) (2024)
- [Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages](https://mlsanity.com/paper/2408.02290/) (2024)

## Abstract (arXiv)

Multilingual Neural Machine Translation approaches are based on the use of task-specific models and the addition of one more language can only be done by retraining the whole system. In this work, we propose a new training schedule that allows the system to scale to more languages without modification of the previous components based on joint training and language-independent encoder/decoder modules allowing for zero-shot translation. This work in progress shows close results to the state-of-the-art in the WMT task.