# An Overview of Cross-media Retrieval: Concepts, Methodologies, Benchmarks and Challenges

arXiv: https://arxiv.org/abs/1704.02223 · HTML: https://mlsanity.com/paper/1704.02223/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2017.

## Built on by (most cited first, 21 of 21)

- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects](https://mlsanity.com/paper/2306.00956/) (2023)
- [Scientific and Technological Information Oriented Semantics-adversarial and Media-adversarial Cross-media Retrieval](https://mlsanity.com/paper/2203.08615/) (2022)
- [Graph Transformers: A Survey](https://mlsanity.com/paper/2407.09777/) (2024)
- [M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining](https://mlsanity.com/paper/2109.04275/) (2021)
- [Towards Video Anomaly Retrieval from Video Anomaly Detection: New Benchmarks and Model](https://mlsanity.com/paper/2307.12545/) (2023)
- [The Curious Layperson: Fine-Grained Image Recognition without Expert Labels](https://mlsanity.com/paper/2111.03651/) (2021)
- [Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognition](https://mlsanity.com/paper/2202.07901/) (2022)
- [Hierarchical Event Memory for Accurate and Low-latency Online Video Temporal Grounding](https://mlsanity.com/paper/2508.04546/) (2025)
- [Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Models](https://mlsanity.com/paper/2509.19994/) (2025)
- [Hierarchical Similarity Learning for Language-based Product Image Retrieval](https://mlsanity.com/paper/2102.09375/) (2021)
- [Dual Learning with Dynamic Knowledge Distillation and Soft Alignment for Partially Relevant Video Retrieval](https://mlsanity.com/paper/2510.12283/) (2025)
- [Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review](https://mlsanity.com/paper/2207.00782/) (2022)
- [Deep Reversible Consistency Learning for Cross-modal Retrieval](https://mlsanity.com/paper/2501.05686/) (2025)
- [Leveraging Auxiliary Information in Text-to-Video Retrieval: A Review](https://mlsanity.com/paper/2505.23952/) (2025)
- [Cross-Media Scientific Research Achievements Retrieval Based on Deep Language Model](https://mlsanity.com/paper/2203.15595/) (2022)
- [Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval](https://mlsanity.com/paper/2209.12491/) (2022)
- [Deep Manifold Hashing: A Divide-and-Conquer Approach for Semi-Paired Unsupervised Cross-Modal Retrieval](https://mlsanity.com/paper/2209.12599/) (2022)
- [Deep Lifelong Cross-modal Hashing](https://mlsanity.com/paper/2304.13357/) (2023)
- [Performance Improvement of Language-Queried Audio Source Separation Based on Caption Augmentation From Large Language Models for DCASE Challenge 2024 Task 9](https://mlsanity.com/paper/2406.11248/) (2024)
- [One-step Multi-view Clustering With Adaptive Low-rank Anchor-graph Learning](https://mlsanity.com/paper/2509.14724/) (2025)

## Abstract (arXiv)

Multimedia retrieval plays an indispensable role in big data utilization. Past efforts mainly focused on single-media retrieval. However, the requirements of users are highly flexible, such as retrieving the relevant audio clips with one query of image. So challenges stemming from the "media gap", which means that representations of different media types are inconsistent, have attracted increasing attention. Cross-media retrieval is designed for the scenarios where the queries and retrieval results are of different media types. As a relatively new research topic, its concepts, methodologies and benchmarks are still not clear in the literatures. To address these issues, we review more than 100 references, give an overview including the concepts, methodologies, major challenges and open issues, as well as build up the benchmarks including datasets and experimental results. Researchers can directly adopt the benchmarks to promptly evaluate their proposed methods. This will help them to focus on algorithm design, rather than the time-consuming compared methods and results. It is noted that we have constructed a new dataset XMedia, which is the first publicly available dataset with up to five media types (text, image, video, audio and 3D model). We believe this overview will attract more researchers to focus on cross-media retrieval and be helpful to them.