mlsanityData to 7 Oct 2026

Language Tags Matter for Zero-Shot Neural Machine Translation

arXiv 2106.07930 ↗First posted 15 Jun 2021cs.CLcited by 29

Not on the map. Why?

Abstract (arXiv)

Multilingual Neural Machine Translation (MNMT) has aroused widespread interest due to its efficiency. An exciting advantage of MNMT models is that they could also translate between unsupervised (zero-shot) language directions. Language tag (LT) strategies are often adopted to indicate the translation directions in MNMT. In this paper, we demonstrate that the LTs are not only indicators for translation directions but also crucial to zero-shot translation qualities. Unfortunately, previous work tends to ignore the importance of LT strategies. We demonstrate that a proper LT strategy could enhance the consistency of semantic representations and alleviate the off-target issue in zero-shot directions. Experimental results show that by ignoring the source language tag (SLT) and adding the target language tag (TLT) to the encoder, the zero-shot translations could achieve a +8 BLEU score difference over other LT strategies in IWSLT17, Europarl, TED talks translation tasks.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 15 Jun 2021.

Builds on

28 of 40 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  8. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  9. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  10. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  11. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  12. 2017Convolutional Sequence to Sequence Learningcited by 550
16 more
  1. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  2. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  3. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  4. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  5. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  6. 2019Investigating Multilingual NMT Representations at Scalecited by 68
  7. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  8. 2019Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlationscited by 61
  9. 2019The Missing Ingredient in Zero-Shot Neural Machine Translationcited by 59
  10. 2016Multi-Source Neural Translationcited by 37
  11. 2020Improving Zero-Shot Translation by Disentangling Positional Informationcited by 35
  12. 2019Improving Zero-shot Translation with Language-Independent Constraintscited by 33
  13. 2018Multilingual Neural Machine Translation with Task-Specific Attentioncited by 26
  14. 2019Consistency by Agreement in Zero-shot Neural Machine Translationcited by 22
  15. 2019Multilingual Neural Machine Translation With Soft Decoupled Encodingcited by 21
  16. 2020A Comprehensive Survey of Multilingual Neural Machine Translationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 7; 2023: 7; 2024: 10; 2025: 3; 2026: 1.

29 papers.

Sort:

By area:

  1. 2022Using natural language prompts for machine translationcited by 27
  2. 2022Examining Scaling and Transfer of Language Model Architectures for Machine Translationcited by 26
  3. 2025Seed-X: Building Strong Multilingual Translation LLM with 7B Parameterscited by 21
  4. 2021Efficient Inference for Multilingual Neural Machine Translationcited by 10
  5. 2023Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularizationcited by 10
  6. 2023Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performancecited by 10
  7. 2022Understanding and Mitigating the Uncertainty in Zero-Shot Translationcited by 9
  8. 2023Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translationcited by 9
  9. 2024AnyTrans: Translate AnyText in the Image with Large Scale Modelscited by 8
  10. 2022Adapting to Non-Centered Languages for Zero-shot Multilingual Translationcited by 6
  11. 2024Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Modelscited by 5
  12. 2022Learning an Artificial Language for Knowledge-Sharing in Multilingual Translationcited by 4
17 more
  1. 2023Exploring the Impact of Layer Normalization for Zero-shot Neural Machine Translationcited by 3
  2. 2023On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translationcited by 3
  3. 2024Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languagescited by 3
  4. 2024Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translationcited by 3
  5. 2024Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Featurescited by 2
  6. 2025Treasure Hunt: Real-time Targeting of the Long Tail using Training-Time Markerscited by 2
  7. 2026OLA: Output Language Alignment in Code-Switched LLM Interactionscited by 2
  8. 2022Informative Language Representation Learning for Massively Multilingual Neural Machine Translationcited by 1
  9. 2023On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Losscited by 1
  10. 2024LCS: A Language Converter Strategy for Zero-Shot Neural Machine Translationcited by 1
  11. 2022The VolcTrans System for WMT22 Multilingual Machine Translation Taskcited by 0
  12. 2023Variable-length Neural Interlingua Representations for Zero-shot Neural Machine Translationcited by 0
  13. 2024How Far Can 100 Samples Go? Unlocking Overall Zero-Shot Multilingual Translation via Tiny Multi-Parallel Datacited by 0
  14. 2024ACT-MNMT Auto-Constriction Turning for Multilingual Neural Machine Translationcited by 0
  15. 2024On the Shortcut Learning in Multilingual Neural Machine Translationcited by 0
  16. 2024Improving Language Transfer Capability of Decoder-only Architecture in Multilingual Neural Machine Translationcited by 0
  17. 2025Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translationcited by 0