mlsanityData to 14 Oct 2026

UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation

arXiv 2207.04900 ↗First posted 11 Jul 2022cs.CL cs.AI cs.LGcited by 10

On the map: Knowledge distillation → Knowledge distillation for LLMs

Abstract (arXiv)

Most translation tasks among languages belong to the zero-resource translation problem where parallel corpora are unavailable. Multilingual neural machine translation (MNMT) enables one-pass translation using shared semantic space for all languages compared to the two-pass pivot translation but often underperforms the pivot-based method. In this paper, we propose a novel method, named as Unified Multilingual Multiple teacher-student Model for NMT (UM4). Our method unifies source-teacher, target-teacher, and pivot-teacher models to guide the student model for the zero-resource translation. The source teacher and target teacher force the student to learn the direct source to target translation by the distilled knowledge on both source and target sides. The monolingual corpus is further leveraged by the pivot-teacher model to enhance the student model. Experimental results demonstrate that our model of 72 directions significantly outperforms previous methods on the WMT benchmark.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 17 of its 26 references that are in the map. First posted on arXiv 11 Jul 2022.

Builds on

17 of 26 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Sequence-Level Knowledge Distillationcited by 801
  4. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  5. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  6. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 242
  7. 2019Multilingual Neural Machine Translation with Knowledge Distillationcited by 95
  8. 2021Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translationcited by 74
  9. 2016Zero-Resource Translation with Multi-Lingual Neural Machine Translationcited by 46
  10. 2020Multi-task Learning for Multilingual Neural Machine Translationcited by 46
  11. 2019Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languagescited by 36
  12. 2021Improving Multilingual Translation by Representation and Gradient Regularizationcited by 31
5 more
  1. 2016Neural Machine Translation with Pivot Languagescited by 25
  2. 2020A Multilingual View of Unsupervised Machine Translationcited by 23
  3. 2017A Teacher-Student Framework for Zero-Resource Neural Machine Translationcited by 22
  4. 2021Distributionally Robust Multilingual Machine Translationcited by 18
  5. 2019Multilingual Neural Machine Translation for Zero-Resource Languagescited by 5

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 4; 2024: 3.

10 papers.

  1. 2022LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translationcited by 21
  2. 2022GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminatorcited by 20
  3. 2024Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Mergingcited by 16
  4. 2023GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Rankingcited by 11
  5. 2022LogLG: Weakly Supervised Log Anomaly Detection via Log-Event Graph Constructioncited by 8
  6. 2024Leveraging Large Language Models for Enhanced NLP Task Performance through Knowledge Distillation and Optimized Training Strategiescited by 6
  7. 2023HanoiT: Enhancing Context-aware Translation via Selective Contextcited by 4
  8. 2023Multilingual Entity and Relation Extraction from Unified to Language-specific Trainingcited by 3
  9. 2023MT4CrossOIE: Multi-stage Tuning for Cross-lingual Open Information Extractioncited by 3
  10. 2024RoNID: New Intent Discovery with Generated-Reliable Labels and Cluster-friendly Representationscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2021Multilingual Neural Machine Translation:Can Linguistic Hierarchies Help?
  2. 2025Multi-Hypothesis Distillation of Multilingual Neural Translation Models for Low-Resource Languages
  3. 2022Unifying the Convergences in Multilingual Neural Machine Translation
  4. 2026Efficient Multilingual Neural Machine Translation via Corpus-Driven Vocabulary Pruning: An English-Arabic Case Study
  5. 2022Improving Simultaneous Machine Translation with Monolingual Data
  6. 2025Multilingual Non-Autoregressive Machine Translation without Knowledge Distillation
  7. 2024Enhancing Low-Resource NMT with a Multilingual Encoder and Knowledge Distillation: A Case Study
  8. 2023Direct Neural Machine Translation with Task-level Mixture of Experts models