# Disentangling the Roles of Target-Side Transfer and Regularization in Multilingual Machine Translation

arXiv: https://arxiv.org/abs/2402.01772 · HTML: https://mlsanity.com/paper/2402.01772/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 32 references that are in the map.
- First posted on arXiv 1 Feb 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges](https://mlsanity.com/paper/1907.05019/) (2019)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [R-Drop: Regularized Dropout for Neural Networks](https://mlsanity.com/paper/2106.14448/) (2021)
- [Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1604.02201/) (2016)
- [Rapid Adaptation of Neural Machine Translation to New Languages](https://mlsanity.com/paper/1808.04189/) (2018)
- [Investigating Multilingual NMT Representations at Scale](https://mlsanity.com/paper/1909.02197/) (2019)
- [On the Inference Calibration of Neural Machine Translation](https://mlsanity.com/paper/2005.00963/) (2020)
- [Trivial Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1809.00357/) (2018)
- [Improving Multilingual Translation by Representation and Gradient Regularization](https://mlsanity.com/paper/2109.04778/) (2021)
- [Causes and Cures for Interference in Multilingual Translation](https://mlsanity.com/paper/2212.07530/) (2022)
- [Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary](https://mlsanity.com/paper/1811.01137/) (2018)
- [Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens](https://mlsanity.com/paper/2305.11550/) (2023)
- [A Comprehensive Survey of Multilingual Neural Machine Translation](https://mlsanity.com/paper/2001.01115/) (2020)
- [Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages](https://mlsanity.com/paper/1811.00383/) (2018)
- [Breaking Down Multilingual Machine Translation](https://mlsanity.com/paper/2110.08130/) (2021)
- [Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation](https://mlsanity.com/paper/2010.01667/) (2020)

## Built on by (most cited first, 2 of 2)

- [The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation](https://mlsanity.com/paper/2505.13090/) (2025)
- [Cross-Lingual Transfer for Machine Translation in Turkic Languages](https://mlsanity.com/paper/2607.29355/) (2026)

## Abstract (arXiv)

Multilingual Machine Translation (MMT) benefits from knowledge transfer across different language pairs. However, improvements in one-to-many translation compared to many-to-one translation are only marginal and sometimes even negligible. This performance discrepancy raises the question of to what extent positive transfer plays a role on the target-side for one-to-many MT. In this paper, we conduct a large-scale study that varies the auxiliary target side languages along two dimensions, i.e., linguistic similarity and corpus size, to show the dynamic impact of knowledge transfer on the main language pairs. We show that linguistically similar auxiliary target languages exhibit strong ability to transfer positive knowledge. With an increasing size of similar target languages, the positive transfer is further enhanced to benefit the main language pairs. Meanwhile, we find distant auxiliary target languages can also unexpectedly benefit main language pairs, even with minimal positive transfer ability. Apart from transfer, we show distant auxiliary target languages can act as a regularizer to benefit translation performance by enhancing the generalization and model inference calibration.