mlsanityData to 7 Oct 2026

Adapting to Non-Centered Languages for Zero-shot Multilingual Translation

arXiv 2209.04138 ↗First posted 9 Sep 2022cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Multilingual neural machine translation can translate unseen language pairs during training, i.e. zero-shot translation. However, the zero-shot translation is always unstable. Although prior works attributed the instability to the domination of central language, e.g. English, we supplement this viewpoint with the strict dependence of non-centered languages. In this work, we propose a simple, lightweight yet effective language-specific modeling method by adapting to non-centered languages and combining the shared information and the language-specific information to counteract the instability of zero-shot translation. Experiments with Transformer on IWSLT17, Europarl, TED talks, and OPUS-100 datasets show that our method not only performs better than strong baselines in centered data conditions but also can easily fit non-centered data conditions. By further investigating the layer attribution, we show that our proposed method can disentangle the coupled representation in the correct direction.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 34 of its 46 references that are in the map. First posted on arXiv 9 Sep 2022.

Builds on

34 of 46 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  8. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  9. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  10. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  11. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  12. 2018Scaling Neural Machine Translationcited by 204
22 more
  1. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  2. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  3. 2018How Important Is a Neuron?cited by 73
  4. 2019Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlationscited by 61
  5. 2019The Missing Ingredient in Zero-Shot Neural Machine Translationcited by 59
  6. 2016Fully Character-Level Neural Machine Translation without Explicit Segmentationcited by 55
  7. 2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
  8. 2020Improving Zero-Shot Translation by Disentangling Positional Informationcited by 35
  9. 2018Parameter Sharing Methods for Multilingual Self-Attentional Translation Modelscited by 33
  10. 2019Improving Zero-shot Translation with Language-Independent Constraintscited by 33
  11. 2021Improving Multilingual Translation by Representation and Gradient Regularizationcited by 31
  12. 2020Complete Multilingual Neural Machine Translationcited by 30
  13. 2018A neural interlingua for multilingual machine translationcited by 29
  14. 2021Language Tags Matter for Zero-Shot Neural Machine Translationcited by 29
  15. 2018Multilingual Neural Machine Translation with Task-Specific Attentioncited by 26
  16. 2020Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoderscited by 23
  17. 2022Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translationcited by 18
  18. 2020Subword Segmentation and a Single Bridge Language Affect Zero-Shot Neural Machine Translationcited by 13
  19. 2022Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translationcited by 7
  20. 2017MMCR4NLP: Multilingual Multiway Corpora Repository for Natural Language Processingcited by 5
  21. 2019Multilingual Neural Machine Translation for Zero-Resource Languagescited by 5
  22. 2021Parameter Differentiation based Multilingual Neural Machine Translationcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3; 2025: 1.

6 papers.

  1. 2023Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performancecited by 10
  2. 2024Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translationcited by 3
  3. 2023Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translationcited by 2
  4. 2024LCS: A Language Converter Strategy for Zero-Shot Neural Machine Translationcited by 1
  5. 2024Improving Language Transfer Capability of Decoder-only Architecture in Multilingual Neural Machine Translationcited by 0
  6. 2025Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translationcited by 0