# Learning Joint Embedding for Cross-Modal Retrieval

arXiv: https://arxiv.org/abs/1908.07673 · HTML: https://mlsanity.com/paper/1908.07673/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2019.

## Built on by (most cited first, 3 of 3)

- [Complete Cross-triplet Loss in Label Space for Audio-visual Cross-modal Retrieval](https://mlsanity.com/paper/2211.03434/) (2022)
- [Metric Learning with Progressive Self-Distillation for Audio-Visual Embedding Learning](https://mlsanity.com/paper/2501.09608/) (2025)
- [Learning Audio-Visual Embeddings with Inferred Latent Interaction Graphs](https://mlsanity.com/paper/2601.11995/) (2026)

## Abstract (arXiv)

A cross-modal retrieval process is to use a query in one modality to obtain relevant data in another modality. The challenging issue of cross-modal retrieval lies in bridging the heterogeneous gap for similarity computation, which has been broadly discussed in image-text, audio-text, and video-text cross-modal multimedia data mining and retrieval. However, the gap in temporal structures of different data modalities is not well addressed due to the lack of alignment relationship between temporal cross-modal structures. Our research focuses on learning the correlation between different modalities for the task of cross-modal retrieval. We have proposed an architecture: Supervised-Deep Canonical Correlation Analysis (S-DCCA), for cross-modal retrieval. In this forum paper, we will talk about how to exploit triplet neural networks (TNN) to enhance the correlation learning for cross-modal retrieval. The experimental result shows the proposed TNN-based supervised correlation learning architecture can get the best result when the data representation extracted by supervised learning.