mlsanityData to 7 Oct 2026

Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization

arXiv 2305.07310 ↗First posted 12 May 2023cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

The multilingual neural machine translation (NMT) model has a promising capability of zero-shot translation, where it could directly translate between language pairs unseen during training. For good transfer performance from supervised directions to zero-shot directions, the multilingual NMT model is expected to learn universal representations across different languages. This paper introduces a cross-lingual consistency regularization, CrossConST, to bridge the representation gap among different languages and boost zero-shot translation performance. The theoretical analysis shows that CrossConST implicitly maximizes the probability distribution for zero-shot translation, and the experimental results on both low-resource and high-resource benchmarks show that CrossConST consistently improves the translation performance. The experimental analysis also proves that CrossConST could close the sentence representation gap and better align the representation space. Given the universality and simplicity of CrossConST, we believe it can serve as a strong baseline for future multilingual NMT research.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

27 of 37 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  4. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  5. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  6. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  7. 2019Massively Multilingual Neural Machine Translationcited by 246
  8. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  9. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  10. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  11. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  12. 2020Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Informationcited by 64
15 more
  1. 2019Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlationscited by 61
  2. 2019The Missing Ingredient in Zero-Shot Neural Machine Translationcited by 59
  3. 2020On Learning Universal Representations Across Languagescited by 58
  4. 2020Improving Zero-Shot Translation by Disentangling Positional Informationcited by 35
  5. 2019Improving Zero-shot Translation with Language-Independent Constraintscited by 33
  6. 2021Improving Multilingual Translation by Representation and Gradient Regularizationcited by 31
  7. 2018A neural interlingua for multilingual machine translationcited by 29
  8. 2021Language Tags Matter for Zero-Shot Neural Machine Translationcited by 29
  9. 2016Neural Machine Translation with Pivot Languagescited by 25
  10. 2021Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoderscited by 24
  11. 2019Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translationcited by 23
  12. 2019Consistency by Agreement in Zero-shot Neural Machine Translationcited by 22
  13. 2021Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variablescited by 14
  14. 2022Bi-SimCut: A Simple Strategy for Boosting Neural Machine Translationcited by 8
  15. 2018Triangular Architecture for Rare Language Translationcited by 7

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 8.

10 papers.

  1. 2024AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Mergingcited by 34
  2. 2024Pushing the Limits of Zero-shot End-to-End Speech Translationcited by 14
  3. 2024Transcending Language Boundaries: Harnessing LLMs for Low-Resource Language Translationcited by 12
  4. 2023Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularizationcited by 7
  5. 2023An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translationcited by 6
  6. 2024How Multilingual Are Large Language Models Fine-Tuned for Translation?cited by 6
  7. 2024Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Modelscited by 5
  8. 2024Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translationcited by 3
  9. 2024LCS: A Language Converter Strategy for Zero-Shot Neural Machine Translationcited by 1
  10. 2024Vision-fused Attack: Advancing Aggressive and Stealthy Adversarial Text against Neural Machine Translationcited by 1