# Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages

arXiv: https://arxiv.org/abs/1909.09524 · HTML: https://mlsanity.com/paper/1909.09524/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 30 of 36)

- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Sustainable Modular Debiasing of Language Models](https://mlsanity.com/paper/2109.03646/) (2021)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Revisiting the Primacy of English in Zero-shot Cross-lingual Transfer](https://mlsanity.com/paper/2106.16171/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task](https://mlsanity.com/paper/2111.02086/) (2021)
- [Language Models are Good Translators](https://mlsanity.com/paper/2106.13627/) (2021)
- [A Survey on Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2107.04239/) (2021)
- [InterTrans: Leveraging Transitive Intermediate Translations to Enhance LLM-based Code Translation](https://mlsanity.com/paper/2411.01063/) (2024)
- [Visual Comparison of Language Model Adaptation](https://mlsanity.com/paper/2208.08176/) (2022)
- [UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation](https://mlsanity.com/paper/2207.04900/) (2022)
- [Revisiting Multilingual Data Mixtures in Language Model Pretraining](https://mlsanity.com/paper/2510.25947/) (2025)
- [Practice Makes a Solver Perfect: Data Augmentation for Math Word Problem Solvers](https://mlsanity.com/paper/2205.00177/) (2022)
- [From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corpora](https://mlsanity.com/paper/2505.14045/) (2025)
- [Investigating Multi-Pivot Ensembling with Massively Multilingual Machine Translation Models](https://mlsanity.com/paper/2311.07439/) (2023)
- [AUGVIC: Exploiting BiText Vicinity for Low-Resource NMT](https://mlsanity.com/paper/2106.05141/) (2021)
- [Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models](https://mlsanity.com/paper/2411.01141/) (2024)
- [SCALE: Synergized Collaboration of Asymmetric Language Translation Engines](https://mlsanity.com/paper/2309.17061/) (2023)
- [Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languages](https://mlsanity.com/paper/2506.00912/) (2025)
- [EdinSaar@WMT21: North-Germanic Low-Resource Multilingual NMT](https://mlsanity.com/paper/2109.14368/) (2021)
- [Triangular Transfer: Freezing the Pivot for Triangular Machine Translation](https://mlsanity.com/paper/2203.09027/) (2022)
- [VAKTA-SETU: A Speech-to-Speech Machine Translation Service in Select Indic Languages](https://mlsanity.com/paper/2305.12518/) (2023)
- [POMP: Probability-driven Meta-graph Prompter for LLMs in Low-resource Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2401.05596/) (2024)
- [Self-Learning for Zero Shot Neural Machine Translation](https://mlsanity.com/paper/2103.05951/) (2021)
- [Language Model Metrics and Procrustes Analysis for Improved Vector Transformation of NLP Embeddings](https://mlsanity.com/paper/2106.02490/) (2021)
- [MALM: Mixing Augmented Language Modeling for Zero-Shot Machine Translation](https://mlsanity.com/paper/2210.00320/) (2022)
- [A Data Selection Approach for Enhancing Low Resource Machine Translation Using Cross-Lingual Sentence Representations](https://mlsanity.com/paper/2409.02712/) (2024)
- [Integrated Training for Sequence-to-Sequence Models Using Non-Autoregressive Transformer](https://mlsanity.com/paper/2109.12950/) (2021)
- [Towards Reinforcement Learning for Pivot-based Neural Machine Translation with Non-autoregressive Transformer](https://mlsanity.com/paper/2109.13097/) (2021)
- [End-to-end Training and Decoding for Pivot-based Cascaded Translation Model](https://mlsanity.com/paper/2305.02261/) (2023)

## Abstract (arXiv)

We present effective pre-training strategies for neural machine translation (NMT) using parallel corpora involving a pivot language, i.e., source-pivot and pivot-target, leading to a significant improvement in source-target translation. We propose three methods to increase the relation among source, pivot, and target languages in the pre-training: 1) step-wise training of a single model for different language pairs, 2) additional adapter component to smoothly connect pre-trained encoder and decoder, and 3) cross-lingual encoder training via autoencoding of the pivot language. Our methods greatly outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. We show that our improvements are valid also in zero-shot/zero-resource scenarios.