# Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary

arXiv: https://arxiv.org/abs/1811.01137 · HTML: https://mlsanity.com/paper/1811.01137/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2018.

## Built on by (most cited first, 18 of 18)

- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Community](https://mlsanity.com/paper/2408.09110/) (2024)
- [Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution](https://mlsanity.com/paper/2103.06799/) (2021)
- [Facilitating large language model Russian adaptation with Learned Embedding Propagation](https://mlsanity.com/paper/2412.21140/) (2024)
- [Order Matters in the Presence of Dataset Imbalance for Multilingual Learning](https://mlsanity.com/paper/2312.06134/) (2023)
- [Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translation](https://mlsanity.com/paper/2306.01382/) (2023)
- [Impact of Tokenization on LLaMa Russian Adaptation](https://mlsanity.com/paper/2312.02598/) (2023)
- [Investigating Massive Multilingual Pre-Trained Machine Translation Models for Clinical Domain via Transfer Learning](https://mlsanity.com/paper/2210.06068/) (2022)
- [Extending Multilingual Machine Translation through Imitation Learning](https://mlsanity.com/paper/2311.08538/) (2023)
- [Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages](https://mlsanity.com/paper/2302.03528/) (2023)
- [Disentangling the Roles of Target-Side Transfer and Regularization in Multilingual Machine Translation](https://mlsanity.com/paper/2402.01772/) (2024)
- [Self-Learning for Zero Shot Neural Machine Translation](https://mlsanity.com/paper/2103.05951/) (2021)
- [SiniticMTError: A Machine Translation Dataset with Error Annotations for Sinitic Languages](https://mlsanity.com/paper/2509.20557/) (2025)
- [The VolcTrans System for WMT22 Multilingual Machine Translation Task](https://mlsanity.com/paper/2210.11599/) (2022)
- [Is Robustness Transferable across Languages in Multilingual Neural Machine Translation?](https://mlsanity.com/paper/2310.20162/) (2023)
- [Accelerating Full Waveform Inversion By Transfer Learning](https://mlsanity.com/paper/2408.00695/) (2024)
- [Low-Resource English-Tigrinya MT: Leveraging Multilingual Models, Custom Tokenizers, and Clean Evaluation Benchmarks](https://mlsanity.com/paper/2509.20209/) (2025)
- [MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation](https://mlsanity.com/paper/2602.21379/) (2026)

## Abstract (arXiv)

We propose a method to transfer knowledge across neural machine translation (NMT) models by means of a shared dynamic vocabulary. Our approach allows to extend an initial model for a given language pair to cover new languages by adapting its vocabulary as long as new data become available (i.e., introducing new vocabulary items if they are not included in the initial model). The parameter transfer mechanism is evaluated in two scenarios: i) to adapt a trained single language NMT system to work with a new language pair and ii) to continuously add new language pairs to grow to a multilingual NMT system. In both the scenarios our goal is to improve the translation performance, while minimizing the training convergence time. Preliminary experiments spanning five languages with different training data sizes (i.e., 5k and 50k parallel sentences) show a significant performance gain ranging from +3.85 up to +13.63 BLEU in different language directions. Moreover, when compared with training an NMT model from scratch, our transfer-learning approach allows us to reach higher performance after training up to 4% of the total training steps.