mlsanityData to 14 Oct 2026

Investigating the Effect of Parallel Data in the Cross-Lingual Transfer for Vision-Language Encoders

arXiv 2504.21681 ↗First posted 30 Apr 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Most pre-trained Vision-Language (VL) models and training data for the downstream tasks are only available in English. Therefore, multilingual VL tasks are solved using cross-lingual transfer: fine-tune a multilingual pre-trained model or transfer the text encoder using parallel data. We study the alternative approach: transferring an already trained encoder using parallel data. We investigate the effect of parallel data: domain and the number of languages, which were out of focus in previous work. Our results show that even machine-translated task data are the best on average, caption-like authentic parallel data outperformed it in some languages. Further, we show that most languages benefit from multilingual training.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 39 references that are in the map. First posted on arXiv 30 Apr 2025.

Builds on

26 of 39 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  7. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  8. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  9. 2019How multilingual is Multilingual BERT?cited by 645
  10. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  11. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  12. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 459
14 more
  1. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  2. 2021Visually Grounded Reasoning across Languages and Culturescited by 189
  3. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  4. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 146
  5. 2020Multilingual Alignment of Contextual Word Representationscited by 114
  6. 2022Masked Vision and Language Modeling for Multi-modal Representation Learningcited by 67
  7. 2021UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-trainingcited by 64
  8. 2022Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generationcited by 62
  9. 2022BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learningcited by 56
  10. 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
  11. 2023mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMscited by 41
  12. 2024M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Taskscited by 27
  13. 2022Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-trainingcited by 24
  14. 2023Stop Pre-Training: Adapt Visual-Language Models to Unseen Languagescited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Multilingual Vision-Language Models, A Surveycited by 1