mlsanityData to 7 Oct 2026

Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens

arXiv 2305.11550 ↗First posted 19 May 2023cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

We argue that translation quality alone is not a sufficient metric for measuring knowledge transfer in multilingual neural machine translation. To support this claim, we introduce Representational Transfer Potential (RTP), which measures representational similarities between languages. We show that RTP can measure both positive and negative transfer (interference), and find that RTP is strongly correlated with changes in translation quality, indicating that transfer does occur. Furthermore, we investigate data and language characteristics that are relevant for transfer, and find that multi-parallel overlap is an important yet under-explored feature. Based on this, we develop a novel training scheme, which uses an auxiliary similarity loss that encourages representations to be more invariant across languages by taking advantage of multi-parallel data. We show that our method yields increased translation quality for low- and mid-resource languages across multiple data and model setups.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 34 of its 46 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

34 of 46 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  6. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  7. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  8. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  9. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  10. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  11. 2016Using the Output Embedding to Improve Language Modelscited by 266
  12. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
22 more
  1. 2019Massively Multilingual Neural Machine Translationcited by 246
  2. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  3. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  4. 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 180
  5. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
  6. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 143
  7. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  8. 2020BLEU might be Guilty but References are not Innocentcited by 109
  9. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  10. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  11. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  12. 2019Investigating Multilingual NMT Representations at Scalecited by 68
  13. 2020Improving Multilingual Models with Language-Clustered Vocabulariescited by 53
  14. 2018Trivial Transfer Learning for Low-Resource Neural Machine Translationcited by 47
  15. 2020Complete Multilingual Neural Machine Translationcited by 30
  16. 2022Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languagescited by 25
  17. 2020Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representationscited by 21
  18. 2019From Bilingual to Multilingual Neural Machine Translation by Incremental Trainingcited by 14
  19. 2021Alternative Input Signals Ease Transfer in Multilingual Machine Translationcited by 14
  20. 2021Back-translation for Large-Scale Multilingual Machine Translationcited by 11
  21. 2021Do Multilingual Neural Machine Translation Models Contain Language Pair Specific Attention Heads?cited by 6
  22. 2022EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 2; 2026: 4.

15 papers.

Sort:

  1. 2023Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performancecited by 10
  2. 2024Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfercited by 6
  3. 2023Multilingual k-Nearest-Neighbor Machine Translationcited by 3
  4. 2024A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translationcited by 3
  5. 2024Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translationcited by 3
  6. 2024Disentangling the Roles of Target-Side Transfer and Regularization in Multilingual Machine Translationcited by 2
  7. 2024Investigating the translation capabilities of Large Language Models trained on parallel data onlycited by 2
  8. 2023UvA-MT's Participation in the WMT23 General Translation Shared Taskcited by 1
  9. 2025The Effect of Language Diversity When Fine-Tuning Large Language Models for Translationcited by 1
  10. 2024Improving Language Transfer Capability of Decoder-only Architecture in Multilingual Neural Machine Translationcited by 0
  11. 2025Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translationcited by 0
  12. 2026Analyzing and Improving Cross-lingual Knowledge Transfer for Machine Translationcited by 0
3 more
  1. 2026The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translationcited by 0
  2. 2026Cross-Lingual Transfer for Machine Translation in Turkic Languagescited by 0
  3. 2026One Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthographycited by 0