The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translation
Not on the map. Why?
Abstract (arXiv)
Knowledge transfer, especially across related languages, has been found beneficial for multilingual neural machine translation (MNMT), but some aspects are still under-explored and deserve further investigation. A joint vocabulary is most often applied to form a uniform word embedding space, but since the impact of a disjoint vocabulary on model performance is far less studied, there is no consensus on how much knowledge transfer is mainly due to vocabulary overlap. In this paper, we present systematic experiments with joint and disjoint vocabularies, and auxiliary languages related and unrelated to the source language. We design this experiment in an out-of-domain setup in order to emphasize transfer and the impact of the auxiliary language. As expected, we yield better results with more extensive vocabulary overlaps typical for related languages, but our experiments also show that domain-match and language relatedness are more important than a joint vocabulary.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 40 references that are in the map. First posted on arXiv 5 May 2026.
Builds on
27 of 40 references.
References · 27
- 2017Attention Is All You Needcited by 59,932
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
- 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
- 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
- 2018Marian: Fast Neural Machine Translation in C++cited by 202
- 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
- 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
- 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
15 more
- 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
- 2017Transfer Learning across Low-Resource, Related Languages for Neural Machine Translationcited by 58
- 2020Improving Multilingual Models with Language-Clustered Vocabulariescited by 53
- 2018Trivial Transfer Learning for Low-Resource Neural Machine Translationcited by 47
- 2021When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfercited by 41
- 2019Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabulariescited by 27
- 2022Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languagescited by 25
- 2020Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoderscited by 23
- 2022Causes and Cures for Interference in Multilingual Translationcited by 23
- 2023Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lenscited by 15
- 2021Alternative Input Signals Ease Transfer in Multilingual Machine Translationcited by 14
- 2023Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translationcited by 9
- 2020Understanding Cross-Lingual Syntactic Transfer in Multilingual Recurrent Neural Networkscited by 7
- 2024A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translationcited by 3
- 2023Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languagescited by 1
Built on by
Not cited yet.