# Alternative Input Signals Ease Transfer in Multilingual Machine Translation

arXiv: https://arxiv.org/abs/2110.07804 · HTML: https://mlsanity.com/paper/2110.07804/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 30 of 31)

- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation](https://mlsanity.com/paper/2107.10821/) (2021)
- [Towards Robust Neural Networks via Random Self-ensemble](https://mlsanity.com/paper/1712.00673/) (2017)
- [Unsupervised Neural Machine Translation](https://mlsanity.com/paper/1710.11041/) (2017)
- [CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs](https://mlsanity.com/paper/1911.06154/) (2019)
- [When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models](https://mlsanity.com/paper/2010.12858/) (2020)
- [Neural Machine Translation with Byte-Level Subwords](https://mlsanity.com/paper/1909.03341/) (2019)
- [Multi-task Sequence to Sequence Learning](https://mlsanity.com/paper/1511.06114/) (2015)
- [The FLoRes Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English](https://mlsanity.com/paper/1902.01382/) (2019)
- [Facebook AI WMT21 News Translation Task Submission](https://mlsanity.com/paper/2108.03265/) (2021)
- [Fully Character-Level Neural Machine Translation without Explicit Segmentation](https://mlsanity.com/paper/1610.03017/) (2016)
- [Zero-Resource Translation with Multi-Lingual Neural Machine Translation](https://mlsanity.com/paper/1606.04164/) (2016)
- [Multi-Source Neural Translation](https://mlsanity.com/paper/1601.00710/) (2016)
- [Character-based Neural Machine Translation](https://mlsanity.com/paper/1603.00810/) (2016)
- [Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation](https://mlsanity.com/paper/2002.10345/) (2020)
- [Role of Language Relatedness in Multilingual Fine-tuning of Language Models: A Case Study in Indo-Aryan Languages](https://mlsanity.com/paper/2109.10534/) (2021)
- [Linguistically Motivated Vocabulary Reduction for Neural Machine Translation from Turkish to English](https://mlsanity.com/paper/1707.09879/) (2017)
- [Input Combination Strategies for Multi-Source Transformer Decoder](https://mlsanity.com/paper/1811.04716/) (2018)
- [Multilingual Neural Machine Translation With Soft Decoupled Encoding](https://mlsanity.com/paper/1902.03499/) (2019)
- [Robust Neural Machine Translation with Joint Textual and Phonetic Embedding](https://mlsanity.com/paper/1810.06729/) (2018)
- [Modeling Source Syntax for Neural Machine Translation](https://mlsanity.com/paper/1705.01020/) (2017)
- [Multi-Source Neural Machine Translation with Missing Data](https://mlsanity.com/paper/1806.02525/) (2018)
- [Orthographic Syllable as basic unit for SMT between Related Languages](https://mlsanity.com/paper/1610.00634/) (2016)
- [Utilizing Language Relatedness to improve Machine Translation: A Case Study on Languages of the Indian Subcontinent](https://mlsanity.com/paper/2003.08925/) (2020)
- [Unsupervised Machine Translation On Dravidian Languages](https://mlsanity.com/paper/2103.15877/) (2021)

## Built on by (most cited first, 14 of 14)

- [Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer](https://mlsanity.com/paper/2305.14280/) (2023)
- [Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution](https://mlsanity.com/paper/2305.17325/) (2023)
- [Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens](https://mlsanity.com/paper/2305.11550/) (2023)
- [Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performance](https://mlsanity.com/paper/2310.10385/) (2023)
- [Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation](https://mlsanity.com/paper/2305.14189/) (2023)
- [Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfer](https://mlsanity.com/paper/2402.14279/) (2024)
- [MC2: Towards Transparent and Culturally-Aware NLP for Minority Languages in China](https://mlsanity.com/paper/2311.08348/) (2023)
- [Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translation](https://mlsanity.com/paper/2309.16599/) (2023)
- [Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languages](https://mlsanity.com/paper/2305.03207/) (2023)
- [UvA-MT's Participation in the WMT23 General Translation Shared Task](https://mlsanity.com/paper/2310.09946/) (2023)
- [Zero-Shot Cross-Lingual NER Using Phonemic Representations for Low-Resource Languages](https://mlsanity.com/paper/2406.16030/) (2024)
- [The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translation](https://mlsanity.com/paper/2605.04196/) (2026)
- [Cross-Lingual Transfer for Machine Translation in Turkic Languages](https://mlsanity.com/paper/2607.29355/) (2026)
- [Universal or Language-Family-Specific Script Unification for Cross-Lingual Transfer? A Case Study on Turkic Languages](https://mlsanity.com/paper/2608.09356/) (2026)

## Abstract (arXiv)

Recent work in multilingual machine translation (MMT) has focused on the potential of positive transfer between languages, particularly cases where higher-resourced languages can benefit lower-resourced ones. While training an MMT model, the supervision signals learned from one language pair can be transferred to the other via the tokens shared by multiple source languages. However, the transfer is inhibited when the token overlap among source languages is small, which manifests naturally when languages use different writing systems. In this paper, we tackle inhibited transfer by augmenting the training data with alternative signals that unify different writing systems, such as phonetic, romanized, and transliterated input. We test these signals on Indic and Turkic languages, two language families where the writing systems differ but languages still share common features. Our results indicate that a straightforward multi-source self-ensemble -- training a model on a mixture of various signals and ensembling the outputs of the same model fed with different signals during inference, outperforms strong ensemble baselines by 1.3 BLEU points on both language families. Further, we find that incorporating alternative inputs via self-ensemble can be particularly effective when training set is small, leading to +5 BLEU when only 5% of the total training data is accessible. Finally, our analysis demonstrates that including alternative signals yields more consistency and translates named entities more accurately, which is crucial for increased factuality of automated systems.