# MARVEL: Unlocking the Multi-Modal Capability of Dense Retrieval via Visual Module Plugin

arXiv: https://arxiv.org/abs/2310.14037 · HTML: https://mlsanity.com/paper/2310.14037/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2023.

## Built on by (most cited first, 22 of 22)

- [GME: Improving Universal Multimodal Retrieval by Multimodal LLMs](https://mlsanity.com/paper/2412.16855/) (2024)
- [mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data](https://mlsanity.com/paper/2502.08468/) (2025)
- [Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.08826/) (2025)
- [A Survey of Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.08748/) (2025)
- [Benchmarking Retrieval-Augmented Generation in Multi-Modal Contexts](https://mlsanity.com/paper/2502.17297/) (2025)
- [UniECS: Unified Multimodal E-Commerce Search Framework with Gated Cross-modal Fusion](https://mlsanity.com/paper/2508.13843/) (2025)
- [WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report](https://mlsanity.com/paper/2608.24053/) (2026)
- [UltraRAG: A Modular and Automated Toolkit for Adaptive Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.08761/) (2025)
- [jina-embeddings-v5-omni: Geometry-preserving Embeddings via Locked Aligned Towers](https://mlsanity.com/paper/2605.08384/) (2026)
- [From Play to Replay: Composed Video Retrieval for Temporally Fine-Grained Videos](https://mlsanity.com/paper/2506.05274/) (2025)
- [MIRe: Enhancing Multimodal Queries Representation via Fusion-Free Modality Interaction for Multimodal Retrieval](https://mlsanity.com/paper/2411.08334/) (2024)
- [MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering](https://mlsanity.com/paper/2511.12142/) (2025)
- [Towards Text-Image Interleaved Retrieval](https://mlsanity.com/paper/2502.12799/) (2025)
- [HotelMatch-LLM: Joint Multi-Task Training of Small and Large Language Models for Efficient Multimodal Hotel Retrieval](https://mlsanity.com/paper/2506.07296/) (2025)
- [Pailitao-VL: Unified Embedding and Reranker for Real-Time Multi-Modal Industrial Search](https://mlsanity.com/paper/2602.13704/) (2026)
- [Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval](https://mlsanity.com/paper/2603.01082/) (2026)
- [Boosting Text-to-Chart Retrieval through Training with Synthesized Semantic Insights](https://mlsanity.com/paper/2505.10043/) (2025)
- [Windsock is Dancing: Adaptive Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.22694/) (2025)
- [Enhancing Multimodal Retrieval via Complementary Information Extraction and Alignment](https://mlsanity.com/paper/2601.04571/) (2026)
- [From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG](https://mlsanity.com/paper/2605.15019/) (2026)
- [Region-R1: Reinforcing Query-Side Region Cropping for Multi-Modal Re-Ranking](https://mlsanity.com/paper/2604.05268/) (2026)
- [MiMIC: Mitigating Visual Modality Collapse in Universal Multimodal Retrieval While Avoiding Semantic Misalignment](https://mlsanity.com/paper/2604.21326/) (2026)

## Abstract (arXiv)

This paper proposes Multi-modAl Retrieval model via Visual modulE pLugin (MARVEL), which learns an embedding space for queries and multi-modal documents to conduct retrieval. MARVEL encodes queries and multi-modal documents with a unified encoder model, which helps to alleviate the modality gap between images and texts. Specifically, we enable the image understanding ability of the well-trained dense retriever, T5-ANCE, by incorporating the visual module's encoded image features as its inputs. To facilitate the multi-modal retrieval tasks, we build the ClueWeb22-MM dataset based on the ClueWeb22 dataset, which regards anchor texts as queries, and extracts the related text and image documents from anchor-linked web pages. Our experiments show that MARVEL significantly outperforms the state-of-the-art methods on the multi-modal retrieval dataset WebQA and ClueWeb22-MM. MARVEL provides an opportunity to broaden the advantages of text retrieval to the multi-modal scenario. Besides, we also illustrate that the language model has the ability to extract image semantics and partly map the image features to the input word embedding space. All codes are available at https://github.com/OpenMatch/MARVEL.