mlsanityData to 7 Oct 2026

Low-resource Neural Machine Translation with Cross-modal Alignment

arXiv 2210.06716 ↗First posted 13 Oct 2022cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

How to achieve neural machine translation with limited parallel data? Existing techniques often rely on large-scale monolingual corpora, which is impractical for some low-resource languages. In this paper, we turn to connect several low-resource languages to a particular high-resource one by additional visual modality. Specifically, we propose a cross-modal contrastive learning method to learn a shared space for all languages, where both a coarse-grained sentence-level objective and a fine-grained token-level one are introduced. Experimental results and further analysis show that our method can effectively learn the cross-modal and cross-lingual alignment with a small amount of image-text pairs and achieves significant improvements over the text-only baseline under both zero-shot and few-shot scenarios.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 48 of its 58 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

48 of 58 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  10. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  11. 2017Word Translation Without Parallel Datacited by 480
  12. 2020Understanding the Behaviour of Contrastive Losscited by 397
36 more
  1. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  2. 2016Multi30K: Multilingual English-German Image Descriptionscited by 250
  3. 2019Massively Multilingual Neural Machine Translationcited by 246
  4. 2020UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learningcited by 239
  5. 2018Scaling Neural Machine Translationcited by 204
  6. 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
  7. 2020Captioning Images Taken by People Who Are Blindcited by 108
  8. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  9. 2020Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Informationcited by 64
  10. 2019Revisiting Low-Resource Neural Machine Translation: A Case Studycited by 61
  11. 2022Cross-modal Contrastive Learning for Speech Translationcited by 61
  12. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  13. 2019Probing the Need for Visual Context in Multimodal Machine Translationcited by 54
  14. 2020A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translationcited by 51
  15. 2017Doubly-Attentive Decoder for Multi-modal Neural Machine Translationcited by 46
  16. 2021Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translationcited by 44
  17. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  18. 2017Incorporating Global Visual Features into Attention-Based Neural Machine Translationcited by 39
  19. 2020Dynamic Context-guided Capsule Network for Multimodal Machine Translationcited by 36
  20. 2017Attention Strategies for Multi-Source Sequence-to-Sequence Learningcited by 32
  21. 2022On Vision Features in Multimodal Machine Translationcited by 31
  22. 2019Distilling Translations with Visual Awarenesscited by 30
  23. 2022Neural Machine Translation with Phrase-Level Universal Visual Representationscited by 26
  24. 2018A Visual Attention Grounding Neural Model for Multimodal Machine Translationcited by 22
  25. 2021Gumbel-Attention for Multi-modal Machine Translationcited by 20
  26. 2016Multimodal Attention for Neural Machine Translationcited by 19
  27. 2021Cross-lingual Visual Pre-training for Multimodal Machine Translationcited by 19
  28. 2016Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Modelscited by 18
  29. 2018CUNI System for the WMT18 Multimodal Translation Taskcited by 13
  30. 2020Emergent Communication Pretraining for Few-Shot Machine Translationcited by 13
  31. 2017An empirical study on the effectiveness of images in Multimodal Neural Machine Translationcited by 10
  32. 2016Zero-resource Machine Translation by Multimodal Encoder-decoder Network with Multimedia Pivotcited by 9
  33. 2017Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translationcited by 8
  34. 2018LIUM-CVC Submissions for WMT18 Multimodal Translation Taskcited by 8
  35. 2019Unsupervised Neural Machine Translation with SMT as Posterior Regularizationcited by 8
  36. 2017LIUM-CVC Submissions for WMT17 Multimodal Translation Taskcited by 7

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 1.

5 papers.

  1. 2025Anatomy of a Machine Learning Ecosystem: 2 Million Models on Hugging Facecited by 12
  2. 2023Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translationcited by 6
  3. 2024A Survey on Multi-modal Machine Translation: Tasks, Methods and Challengescited by 6
  4. 2022Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mappingscited by 4
  5. 2024Multi-modal Semantic Understanding with Contrastive Cross-modal Feature Alignmentcited by 3