mlsanityData to 14 Oct 2026

HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translation

arXiv 2207.04906 ↗First posted 11 Jul 2022cs.CL cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Multilingual neural machine translation (MNMT) trained in multiple language pairs has attracted considerable attention due to fewer model parameters and lower training costs by sharing knowledge among multiple languages. Nonetheless, multilingual training is plagued by language interference degeneration in shared parameters because of the negative interference among different translation directions, especially on high-resource languages. In this paper, we propose the multilingual translation model with the high-resource language-specific training (HLT-MT) to alleviate the negative interference, which adopts the two-stage training with the language-specific selection mechanism. Specifically, we first train the multilingual model only with the high-resource pairs and select the language-specific modules at the top of the decoder to enhance the translation quality of high-resource directions. Next, the model is further trained on all available corpora to transfer knowledge from high-resource languages (HRLs) to low-resource languages (LRLs). Experimental results show that HLT-MT outperforms various strong baselines on WMT-10 and OPUS-100 benchmarks. Furthermore, the analytic experiments validate the effectiveness of our method in mitigating the negative interference in multilingual training.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 19 of its 24 references that are in the map. First posted on arXiv 11 Jul 2022.

Builds on

19 of 24 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  4. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  5. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  6. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  7. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  8. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  9. 2019Massively Multilingual Neural Machine Translationcited by 246
  10. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  11. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  12. 2020Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Informationcited by 64
7 more
  1. 2019Multilingual Neural Machine Translation with Language Clusteringcited by 47
  2. 2020Multi-task Learning for Multilingual Neural Machine Translationcited by 46
  3. 2020XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoderscited by 24
  4. 2020Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoderscited by 23
  5. 2018Multilingual NMT with a language-independent attention bridgecited by 20
  6. 2022Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoderscited by 12
  7. 2021Adaptive Sparse Transformer for Multilingual Translationcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2.

4 papers.

  1. 2022GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminatorcited by 20
  2. 2023Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systemscited by 6
  3. 2023HanoiT: Enhancing Context-aware Translation via Selective Contextcited by 4
  4. 2022University of Cape Town's WMT22 System: Multilingual Machine Translation for Southern African Languagescited by 0