When do Contrastive Word Alignments Improve Many-to-many Neural Machine Translation?
Not on the map. Why?
Abstract (arXiv)
Word alignment has proven to benefit many-to-many neural machine translation (NMT). However, high-quality ground-truth bilingual dictionaries were used for pre-editing in previous methods, which are unavailable for most language pairs. Meanwhile, the contrastive objective can implicitly utilize automatically learned word alignment, which has not been explored in many-to-many NMT. This work proposes a word-level contrastive objective to leverage word alignments for many-to-many NMT. Empirical results show that this leads to 0.8 BLEU gains for several language pairs. Analyses reveal that in many-to-many NMT, the encoder's sentence retrieval performance highly correlates with the translation quality, which explains when the proposed method impacts translation. This motivates future exploration for many-to-many NMT to improve the encoder's sentence retrieval performance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 35 references that are in the map. First posted on arXiv 26 Apr 2022.
Builds on
21 of 35 references.
References · 21
- 2017Attention Is All You Needcited by 60,361
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
- 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
- 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
- 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
- 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 276
- 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
- 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 251
- 2019Massively Multilingual Neural Machine Translationcited by 246
- 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
9 more
- 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
- 2016Edinburgh Neural Machine Translation Systems for WMT 16cited by 76
- 2019Jointly Learning to Align and Translate with Transformer Modelscited by 75
- 2020Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Informationcited by 64
- 2019The Missing Ingredient in Zero-Shot Neural Machine Translationcited by 59
- 2020On Learning Universal Representations Across Languagescited by 58
- 2020Multilingual BERT Post-Pretraining Alignmentcited by 30
- 2019word2word: A Collection of Bilingual Lexicons for 3,564 Language Pairscited by 7
- 2021Lightweight Cross-Lingual Sentence Representation Learningcited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.