mlsanityData to 7 Oct 2026

Causes and Cures for Interference in Multilingual Translation

arXiv 2212.07530 ↗First posted 14 Dec 2022cs.CL cs.AI cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Multilingual machine translation models can benefit from synergy between different language pairs, but also suffer from interference. While there is a growing number of sophisticated methods that aim to eliminate interference, our understanding of interference as a phenomenon is still limited. This work identifies the main factors that contribute to interference in multilingual machine translation. Through systematic experimentation, we find that interference (or synergy) are primarily determined by model size, data size, and the proportion of each language pair within the total dataset. We observe that substantial interference occurs mainly when the model is very small with respect to the available training data, and that using standard transformer configurations with less than one billion parameters largely alleviates interference and promotes synergy. Moreover, we show that tuning the sampling temperature to control the proportion of each language pair in the data is key to balancing the amount of interference between low and high resource language pairs effectively, and can lead to superior performance overall.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 14 Dec 2022.

Builds on

31 of 37 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  8. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  9. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  10. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  11. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  12. 2016Using the Output Embedding to Improve Language Modelscited by 266
19 more
  1. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  2. 2019Massively Multilingual Neural Machine Translationcited by 246
  3. 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 167
  4. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 143
  5. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  6. 2021Scaling Laws for Neural Machine Translationcited by 110
  7. 2021Larger-Scale Transformers for Multilingual Masked Language Modelingcited by 86
  8. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  9. 2020Balancing Training for Multilingual Neural Machine Translationcited by 71
  10. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 65
  11. 2022Do Current Multi-Task Optimization Methods in Deep Learning Even Help?cited by 60
  12. 2022Data Scaling Laws in NMT: The Effect of Noise and Architecturecited by 55
  13. 2020Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translationcited by 51
  14. 2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
  15. 2021Counter-Interference Adapter for Multilingual Machine Translationcited by 39
  16. 2020Complete Multilingual Neural Machine Translationcited by 30
  17. 2019Adaptive Scheduling for Multi-Task Learningcited by 24
  18. 2021Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Banditscited by 16
  19. 2021Robust Optimization for Multilingual Translation with Imbalanced Datacited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 8; 2025: 3; 2026: 3.

23 papers.

Sort:

By area:

  1. 2023The Falcon Series of Open Language Modelscited by 526
  2. 2023Scaling Laws for Generative Mixed-Modal Language Modelscited by 131
  3. 2024Multilingual Instruction Tuning With Just a Pinch of Multilingualitycited by 71
  4. 2023From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Predictioncited by 39
  5. 2023Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed?cited by 39
  6. 2024Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metriccited by 23
  7. 2023Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfercited by 22
  8. 2024Neuron Specialization: Leveraging intrinsic task modularity for multilingual machine translationcited by 19
  9. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformerscited by 14
  10. 2025Vintix: Action Model via In-Context Reinforcement Learningcited by 8
  11. 2023On the Pareto Front of Multilingual Neural Machine Translationcited by 7
  12. 2024Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasetscited by 4
11 more
  1. 2024Conditional [MASK] Discrete Diffusion Language Modelcited by 4
  2. 2023Condensing Multilingual Knowledge with Lightweight Language-Specific Modulescited by 3
  3. 2023Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Modelcited by 3
  4. 2024A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translationcited by 3
  5. 2024Disentangling the Roles of Target-Side Transfer and Regularization in Multilingual Machine Translationcited by 2
  6. 2023Exploring Representational Disparities Between Multilingual and Bilingual Translation Modelscited by 0
  7. 2025Uniform Loss vs. Specialized Optimization: A Comparative Analysis in Multi-Task Learningcited by 0
  8. 2025LLaVA-NeuMT: Selective Layer-Neuron Modulation for Efficient Multilingual Multimodal Translationcited by 0
  9. 2026The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translationcited by 0
  10. 2026Pretraining Language Models with Subword Regularization: An Empirical Study of BPE Dropout in Low-Resource NLPcited by 0
  11. 2026There is No Theoretical Curse of Multilinguality For Embedding Space Structurecited by 0