# A Teacher-Student Framework for Zero-Resource Neural Machine Translation

arXiv: https://arxiv.org/abs/1705.00753 · HTML: https://mlsanity.com/paper/1705.00753/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2017.

## Built on by (most cited first, 22 of 22)

- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Contrastive Learning for Many-to-many Multilingual Neural Machine Translation](https://mlsanity.com/paper/2105.09501/) (2021)
- [Autoregressive Knowledge Distillation through Imitation Learning](https://mlsanity.com/paper/2009.07253/) (2020)
- [Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders](https://mlsanity.com/paper/2104.08757/) (2021)
- [Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data](https://mlsanity.com/paper/2105.15071/) (2021)
- [A Survey on Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2107.04239/) (2021)
- [UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation](https://mlsanity.com/paper/2207.04900/) (2022)
- [Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation](https://mlsanity.com/paper/2110.08547/) (2021)
- [AnyTrans: Translate AnyText in the Image with Large Scale Models](https://mlsanity.com/paper/2406.11432/) (2024)
- [Token-wise Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2103.11088/) (2021)
- [ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2212.04262/) (2022)
- [SCALE: Synergized Collaboration of Asymmetric Language Translation Engines](https://mlsanity.com/paper/2309.17061/) (2023)
- [Jointly Learning to Repair Code and Generate Commit Message](https://mlsanity.com/paper/2109.12296/) (2021)
- [ZRIGF: An Innovative Multimodal Framework for Zero-Resource Image-Grounded Dialogue Generation](https://mlsanity.com/paper/2308.00400/) (2023)
- [KD4MT: A Survey of Knowledge Distillation for Machine Translation](https://mlsanity.com/paper/2602.15845/) (2026)
- [Self-Learning for Zero Shot Neural Machine Translation](https://mlsanity.com/paper/2103.05951/) (2021)
- [Multilingual Non-Autoregressive Machine Translation without Knowledge Distillation](https://mlsanity.com/paper/2502.04537/) (2025)
- [Improving Zero-shot Multilingual Neural Machine Translation for Low-Resource Languages](https://mlsanity.com/paper/2110.00712/) (2021)
- [Look Backward and Forward: Self-Knowledge Distillation with Bidirectional Decoder for Neural Machine Translation](https://mlsanity.com/paper/2203.05248/) (2022)
- [MSSRNet: Manipulating Sequential Style Representation for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2306.07994/) (2023)
- [Exploring Linguistic Similarity and Zero-Shot Learning for Multilingual Translation of Dravidian Languages](https://mlsanity.com/paper/2308.05574/) (2023)
- [Direct Neural Machine Translation with Task-level Mixture of Experts models](https://mlsanity.com/paper/2310.12236/) (2023)

## Abstract (arXiv)

While end-to-end neural machine translation (NMT) has made remarkable progress recently, it still suffers from the data scarcity problem for low-resource language pairs and domains. In this paper, we propose a method for zero-resource NMT by assuming that parallel sentences have close probabilities of generating a sentence in a third language. Based on this assumption, our method is able to train a source-to-target NMT model ("student") without parallel corpora available, guided by an existing pivot-to-target NMT model ("teacher") on a source-pivot parallel corpus. Experimental results show that the proposed method significantly improves over a baseline pivot-based model by +3.0 BLEU points across various language pairs.