# Cross-Modal Retrieval: A Systematic Review of Methods and Future Directions

arXiv: https://arxiv.org/abs/2308.14263 · HTML: https://mlsanity.com/paper/2308.14263/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2023.

## Built on by (most cited first, 30 of 40)

- [A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions](https://mlsanity.com/paper/2410.12837/) (2024)
- [A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1](https://mlsanity.com/paper/2503.10635/) (2025)
- [Modality Curation: Building Universal Embeddings for Advanced Multimodal Information Retrieval](https://mlsanity.com/paper/2505.19650/) (2025)
- [DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning](https://mlsanity.com/paper/2509.23883/) (2025)
- [Masked Contrastive Reconstruction for Cross-modal Medical Image-Report Retrieval](https://mlsanity.com/paper/2312.15840/) (2023)
- [Language-Driven Visual Consensus for Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2403.08426/) (2024)
- [Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmark](https://mlsanity.com/paper/2507.15882/) (2025)
- [Supervised Fine-Tuning or Contrastive Learning? Towards Better Multimodal LLM Reranking](https://mlsanity.com/paper/2510.14824/) (2025)
- [Pushing the Frontier of Black-Box LVLM Attacks via Fine-Grained Detail Targeting](https://mlsanity.com/paper/2602.17645/) (2026)
- [JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images](https://mlsanity.com/paper/2409.12953/) (2024)
- [Multimodal Classification via Modal-Aware Interactive Enhancement](https://mlsanity.com/paper/2407.04587/) (2024)
- [AnomalyLMM: Bridging Generative Knowledge and Discriminative Retrieval for Text-Based Person Anomaly Search](https://mlsanity.com/paper/2509.04376/) (2025)
- [No Modality Left Behind: Adapting to Missing Modalities via Knowledge Distillation for Brain Tumor Segmentation](https://mlsanity.com/paper/2509.15017/) (2025)
- [Recall: Empowering Multimodal Embedding for Edge Devices](https://mlsanity.com/paper/2409.15342/) (2024)
- [GMM-Based Comprehensive Feature Extraction and Relative Distance Preservation For Few-Shot Cross-Modal Retrieval](https://mlsanity.com/paper/2505.13306/) (2025)
- [WISER: Wider Search, Deeper Thinking, and Adaptive Fusion for Training-Free Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2602.23029/) (2026)
- [Multimodal Data Storage and Retrieval for Embodied AI: A Survey](https://mlsanity.com/paper/2508.13901/) (2025)
- [On the Problem of Consistent Anomalies in Zero-Shot Industrial Anomaly Detection](https://mlsanity.com/paper/2510.10456/) (2025)
- [GeoAlignCLIP: Enhancing Fine-Grained Vision-Language Alignment in Remote Sensing via Multi-Granular Consistency Learning](https://mlsanity.com/paper/2603.09566/) (2026)
- [Complementarity-Preserving Generative Theory for Multimodal ECG Synthesis: A Quantum-Inspired Approach](https://mlsanity.com/paper/2603.26695/) (2026)
- [PARIC: Probabilistic Attention Regularization for Language Guided Image Classification from Pre-trained Vison Language Models](https://mlsanity.com/paper/2503.11360/) (2025)
- [Improving Applicability of Deep Learning based Token Classification models during Training](https://mlsanity.com/paper/2504.01028/) (2025)
- [The Curious Case of Visual Grounding: Different Effects for Speech- and Text-based Language Encoders](https://mlsanity.com/paper/2509.15837/) (2025)
- [Deep Learning-based Cross-modal Reconstruction of Vehicle Target from Sparse 3D SAR Image](https://mlsanity.com/paper/2406.04158/) (2024)
- [FiCo-ITR: bridging fine-grained and coarse-grained image-text retrieval for comparative performance analysis](https://mlsanity.com/paper/2407.20114/) (2024)
- [An Enhanced Large Language Model For Cross Modal Query Understanding System Using DL-KeyBERT Based CAZSSCL-MPGPT](https://mlsanity.com/paper/2502.17000/) (2025)
- [Multimodal Federated Learning: A Survey through the Lens of Different FL Paradigms](https://mlsanity.com/paper/2505.21792/) (2025)
- [Describe-to-Score: A text-guided framework for image complexity assessment](https://mlsanity.com/paper/2509.16609/) (2025)
- [Cross-attention Secretly Performs Orthogonal Alignment in Recommendation Models](https://mlsanity.com/paper/2510.09435/) (2025)
- [Information-Theoretic Criteria for Knowledge Distillation in Multimodal Learning](https://mlsanity.com/paper/2510.13182/) (2025)

## Abstract (arXiv)

With the exponential surge in diverse multi-modal data, traditional uni-modal retrieval methods struggle to meet the needs of users seeking access to data across various modalities. To address this, cross-modal retrieval has emerged, enabling interaction across modalities, facilitating semantic matching, and leveraging complementarity and consistency between heterogeneous data. Although prior literature has reviewed the field of cross-modal retrieval, it suffers from numerous deficiencies in terms of timeliness, taxonomy, and comprehensiveness. This paper conducts a comprehensive review of cross-modal retrieval's evolution, spanning from shallow statistical analysis techniques to vision-language pre-training models. Commencing with a comprehensive taxonomy grounded in machine learning paradigms, mechanisms, and models, the paper delves deeply into the principles and architectures underpinning existing cross-modal retrieval methods. Furthermore, it offers an overview of widely-used benchmarks, metrics, and performances. Lastly, the paper probes the prospects and challenges that confront contemporary cross-modal retrieval, while engaging in a discourse on potential directions for further progress in the field. To facilitate the ongoing research on cross-modal retrieval, we develop a user-friendly toolbox and an open-source repository at https://cross-modal-retrieval.github.io.