# Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies

arXiv: https://arxiv.org/abs/1905.05475 · HTML: https://mlsanity.com/paper/1905.05475/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2019.

## Built on by (most cited first, 27 of 27)

- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Contrastive Learning for Many-to-many Multilingual Neural Machine Translation](https://mlsanity.com/paper/2105.09501/) (2021)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Improving Fake News Detection of Influential Domain via Domain- and Instance-Level Transfer](https://mlsanity.com/paper/2209.08902/) (2022)
- [A Survey on Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2107.04239/) (2021)
- [Back-translation for Large-Scale Multilingual Machine Translation](https://mlsanity.com/paper/2109.08712/) (2021)
- [Krey`ol-MT: Building MT for Latin American, Caribbean and Colonial African Creole Languages](https://mlsanity.com/paper/2405.05376/) (2024)
- [Transfer Language Selection for Zero-Shot Cross-Lingual Abusive Language Detection](https://mlsanity.com/paper/2206.00962/) (2022)
- [Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation](https://mlsanity.com/paper/2504.01919/) (2025)
- [Order Matters in the Presence of Dataset Imbalance for Multilingual Learning](https://mlsanity.com/paper/2312.06134/) (2023)
- [Is Encoder-Decoder Redundant for Neural Machine Translation?](https://mlsanity.com/paper/2210.11807/) (2022)
- [Improving Neural Machine Translation of Indigenous Languages with Multilingual Transfer Learning](https://mlsanity.com/paper/2205.06993/) (2022)
- [ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2212.04262/) (2022)
- [Cross-lingual Transferring of Pre-trained Contextualized Language Models](https://mlsanity.com/paper/2107.12627/) (2021)
- [Focus on the Target's Vocabulary: Masked Label Smoothing for Machine Translation](https://mlsanity.com/paper/2203.02889/) (2022)
- [Sub-Word Alignment Is Still Useful: A Vest-Pocket Method for Enhancing Low-Resource Machine Translation](https://mlsanity.com/paper/2205.04067/) (2022)
- [Unveiling the Influence of Amplifying Language-Specific Neurons](https://mlsanity.com/paper/2507.22581/) (2025)
- [Enhanced Simultaneous Machine Translation with Word-level Policies](https://mlsanity.com/paper/2310.16417/) (2023)
- [The Impact of Syntactic and Semantic Proximity on Machine Translation with Back-Translation](https://mlsanity.com/paper/2403.18031/) (2024)
- [Unsupervised Transfer Learning in Multilingual Neural Machine Translation with Cross-Lingual Word Embeddings](https://mlsanity.com/paper/2103.06689/) (2021)
- [Self-Attentive Constituency Parsing for UCCA-based Semantic Parsing](https://mlsanity.com/paper/2110.00621/) (2021)
- [Benchmarking Azerbaijani Neural Machine Translation](https://mlsanity.com/paper/2207.14473/) (2022)
- [University of Cape Town's WMT22 System: Multilingual Machine Translation for Southern African Languages](https://mlsanity.com/paper/2210.11757/) (2022)
- [Ngambay-French Neural Machine Translation (sba-Fr)](https://mlsanity.com/paper/2308.13497/) (2023)
- [Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages](https://mlsanity.com/paper/2408.02290/) (2024)
- [The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translation](https://mlsanity.com/paper/2605.04196/) (2026)
- [Translation as a Decision Space: A Multi-Agent Perspective on Low-Resource Dialect Generation](https://mlsanity.com/paper/2609.04048/) (2026)

## Abstract (arXiv)

Transfer learning or multilingual model is essential for low-resource neural machine translation (NMT), but the applicability is limited to cognate languages by sharing their vocabularies. This paper shows effective techniques to transfer a pre-trained NMT model to a new, unrelated language without shared vocabularies. We relieve the vocabulary mismatch by using cross-lingual word embedding, train a more language-agnostic encoder by injecting artificial noises, and generate synthetic data easily from the pre-training data without back-translation. Our methods do not require restructuring the vocabulary or retraining the model. We improve plain NMT transfer by up to +5.1% BLEU in five low-resource translation tasks, outperforming multilingual joint training by a large margin. We also provide extensive ablation studies on pre-trained embedding, synthetic data, vocabulary size, and parameter freezing for a better understanding of NMT transfer.