mlsanityData to 7 Oct 2026

Towards Zero-shot Cross-lingual Image Retrieval

arXiv 2012.05107 ↗First posted 24 Nov 2020cs.CL cs.CV cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

There has been a recent spike in interest in multi-modal Language and Vision problems. On the language side, most of these models primarily focus on English since most multi-modal datasets are monolingual. We try to bridge this gap with a zero-shot approach for learning multi-modal representations using cross-lingual pre-training on the text side. We present a simple yet practical approach for building a cross-lingual image retrieval model which trains on a monolingual training dataset but can be used in a zero-shot cross-lingual fashion during inference. We also introduce a new objective function which tightens the text embedding clusters by pushing dissimilar texts from each other. Finally, we introduce a new 1K multi-lingual MSCOCO2014 caption test dataset (XTD10) in 7 languages that we collected using a crowdsourcing platform. We use this as the test set for evaluating zero-shot model performance across languages. XTD10 dataset is made publicly available here: https://github.com/adobe-research/Cross-lingual-Test-Dataset-XTD10

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 9; 2024: 3; 2025: 5; 2026: 2.

25 papers.

Sort:

By area:

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 84
  3. 2025Meta CLIP 2: A Worldwide Scaling Recipecited by 59
  4. 2025mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Datacited by 49
  5. 2021MURAL: Multimodal, Multitask Retrieval Across Languagescited by 44
  6. 2024Do Vision and Language Encoders Represent the World Similarly?cited by 44
  7. 2024jina-clip-v2: Multilingual Multimodal Embeddings for Text and Imagescited by 36
  8. 2023AltDiffusion: A Multilingual Text-to-Image Diffusion Modelcited by 30
  9. 2023NLLB-CLIP -- train performant multilingual image retrieval model on a budgetcited by 24
  10. 2022Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learningcited by 22
  11. 2023Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learningcited by 16
  12. 2023Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representationscited by 11
13 more
  1. 2024Harnessing Frozen Unimodal Encoders for Flexible Multimodal Alignmentcited by 11
  2. 2022MaXM: Towards Multilingual Visual Question Answeringcited by 9
  3. 2023Translation-Enhanced Multilingual Text-to-Image Generationcited by 9
  4. 2023Dual-view Curricular Optimal Transport for Cross-lingual Cross-modal Retrievalcited by 9
  5. 2023PEA-Diffusion: Parameter-Efficient Adapter with Knowledge Distillation in non-English Text-to-Image Generationcited by 5
  6. 2021Towards Zero-shot Cross-lingual Image Retrieval and Taggingcited by 4
  7. 2022Generalizing Multimodal Pre-training into Multilingual via Language Acquisitioncited by 3
  8. 2025xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillationcited by 3
  9. 2025Multilingual Text-to-Image Person Retrieval via Bidirectional Relation Reasoning and Aligningcited by 3
  10. 2023Parameter-Efficient Cross-lingual Transfer of Vision and Language Models via Translation-based Alignmentcited by 2
  11. 2025Multilingual Vision-Language Models, A Surveycited by 1
  12. 2026Multilingual-To-Multimodal (M2M): Unlocking New Languages with Monolingual Textcited by 0
  13. 2026Where Do Multilingual Vision-Language Encoders Fail on Low-Resource Languages?cited by 0