# Image search using multilingual texts: a cross-modal learning approach between image and text

arXiv: https://arxiv.org/abs/1903.11299 · HTML: https://mlsanity.com/paper/1903.11299/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2019.

## Built on by (most cited first, 7 of 7)

- [AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities](https://mlsanity.com/paper/2211.06679/) (2022)
- [Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learning](https://mlsanity.com/paper/2208.12526/) (2022)
- [Universal Multimodal Representation for Language Understanding](https://mlsanity.com/paper/2301.03344/) (2023)
- [Dual-view Curricular Optimal Transport for Cross-lingual Cross-modal Retrieval](https://mlsanity.com/paper/2309.05451/) (2023)
- [Towards Zero-shot Cross-lingual Image Retrieval and Tagging](https://mlsanity.com/paper/2109.07622/) (2021)
- [Multilingual Text-to-Image Person Retrieval via Bidirectional Relation Reasoning and Aligning](https://mlsanity.com/paper/2510.17685/) (2025)
- [Dynamic Adapter with Semantics Disentangling for Cross-lingual Cross-modal Retrieval](https://mlsanity.com/paper/2412.13510/) (2024)

## Abstract (arXiv)

Multilingual (or cross-lingual) embeddings represent several languages in a unique vector space. Using a common embedding space enables for a shared semantic between words from different languages. In this paper, we propose to embed images and texts into a unique distributional vector space, enabling to search images by using text queries expressing information needs related to the (visual) content of images, as well as using image similarity. Our framework forces the representation of an image to be similar to the representation of the text that describes it. Moreover, by using multilingual embeddings we ensure that words from two different languages have close descriptors and thus are attached to similar images. We provide experimental evidence of the efficiency of our approach by experimenting it on two datasets: Common Objects in COntext (COCO) [19] and Multi30K [7].