mlsanityData to 7 Oct 2026

A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation

arXiv 2403.20157 ↗First posted 29 Mar 2024cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Multilingual modelling can improve machine translation for low-resource languages, partly through shared subword representations. This paper studies the role of subword segmentation in cross-lingual transfer. We systematically compare the efficacy of several subword methods in promoting synergy and preventing interference across different linguistic typologies. Our findings show that subword regularisation boosts synergy in multilingual modelling, whereas BPE more effectively facilitates transfer during cross-lingual fine-tuning. Notably, our results suggest that differences in orthographic word boundary conventions (the morphological granularity of written words) may impede cross-lingual transfer more significantly than linguistic unrelatedness. Our study confirms that decisions around subword modelling can be key to optimising the benefits of multilingual modelling.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 29 Mar 2024.

Builds on

20 of 27 references.

References · 20

  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  3. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  4. 2019How multilingual is Multilingual BERT?cited by 642
  5. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  6. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  7. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 313
  8. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  9. 2019Massively Multilingual Neural Machine Translationcited by 246
  10. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  11. 2019Neural Machine Translation with Byte-Level Subwordscited by 115
  12. 2020Extending Multilingual BERT to Low-Resource Languagescited by 88
8 more
  1. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  2. 2020Improving Multilingual Models with Language-Clustered Vocabulariescited by 53
  3. 2021Multi-view Subword Regularizationcited by 40
  4. 2022Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languagescited by 25
  5. 2022Causes and Cures for Interference in Multilingual Translationcited by 23
  6. 2023Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lenscited by 15
  7. 2021Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languagescited by 8
  8. 2023Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translationcited by 0
  2. 2026Pretraining Language Models with Subword Regularization: An Empirical Study of BPE Dropout in Low-Resource NLPcited by 0
  3. 2026One Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthographycited by 0