# Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval

arXiv: https://arxiv.org/abs/2602.19961 · HTML: https://mlsanity.com/paper/2602.19961/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 180 of its 230 references that are in the map.
- First posted on arXiv 23 Feb 2026.

## Builds on (most cited first, 30 of 180)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [WebThinker: Empowering Large Reasoning Models with Deep Research Capability](https://mlsanity.com/paper/2504.21776/) (2025)
- [Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG](https://mlsanity.com/paper/2501.09136/) (2025)
- [VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents](https://mlsanity.com/paper/2410.10594/) (2024)
- [Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking](https://mlsanity.com/paper/2601.04720/) (2026)
- [VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks](https://mlsanity.com/paper/2410.05160/) (2024)
- [When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method](https://mlsanity.com/paper/2402.17193/) (2024)
- [GME: Improving Universal Multimodal Retrieval by Multimodal LLMs](https://mlsanity.com/paper/2412.16855/) (2024)
- [UniIR: Training and Benchmarking Universal Multimodal Information Retrievers](https://mlsanity.com/paper/2311.17136/) (2023)
- [MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs](https://mlsanity.com/paper/2411.02571/) (2024)
- [BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval](https://mlsanity.com/paper/2407.12883/) (2024)
- [Learning Multi-Level Features with Matryoshka Sparse Autoencoders](https://mlsanity.com/paper/2503.17547/) (2025)
- [Unifying Vision, Text, and Layout for Universal Document Processing](https://mlsanity.com/paper/2212.02623/) (2022)
- [EmbeddingGemma: Powerful and Lightweight Text Representations](https://mlsanity.com/paper/2509.20354/) (2025)
- [Revisiting Neural Scaling Laws in Language and Vision](https://mlsanity.com/paper/2209.06640/) (2022)
- [MMTEB: Massive Multilingual Text Embedding Benchmark](https://mlsanity.com/paper/2502.13595/) (2025)
- [M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding](https://mlsanity.com/paper/2411.04952/) (2024)
- [Unifying Multimodal Retrieval via Document Screenshot Embedding](https://mlsanity.com/paper/2406.11251/) (2024)
- [Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts](https://mlsanity.com/paper/2405.11273/) (2024)
- [LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant](https://mlsanity.com/paper/2412.01720/) (2024)
- [InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales](https://mlsanity.com/paper/2406.13629/) (2024)
- [CodeRAG-Bench: Can Retrieval Augment Code Generation?](https://mlsanity.com/paper/2406.14497/) (2024)
- [Retrieving Multimodal Information for Augmented Generation: A Survey](https://mlsanity.com/paper/2303.10868/) (2023)
- [MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval](https://mlsanity.com/paper/2412.14475/) (2024)
- [Towards Complex Document Understanding By Discrete Reasoning](https://mlsanity.com/paper/2207.11871/) (2022)
- [jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval](https://mlsanity.com/paper/2506.18902/) (2025)
- [Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs](https://mlsanity.com/paper/2504.17432/) (2025)
- [Retrieval-Augmented Generation for Natural Language Processing: A Survey](https://mlsanity.com/paper/2407.13193/) (2024)

## Built on by (most cited first, 8 of 8)

- [Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval](https://mlsanity.com/paper/2604.10167/) (2026)
- [Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations](https://mlsanity.com/paper/2603.01666/) (2026)
- [MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework](https://mlsanity.com/paper/2606.07654/) (2026)
- [PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation](https://mlsanity.com/paper/2606.28344/) (2026)
- [AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrieval](https://mlsanity.com/paper/2608.08732/) (2026)
- [VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval](https://mlsanity.com/paper/2608.17889/) (2026)
- [KoViDoRe: A Benchmark for Korean Visual Document Retrieval](https://mlsanity.com/paper/2608.20840/) (2026)
- [Spatial Matryoshka Training for Multi-Granularity Visual Document Retrieval](https://mlsanity.com/paper/2608.29951/) (2026)

## Abstract (arXiv)

With the rapid proliferation of multimodal information, Visual Document Retrieval (VDR) has emerged as a critical frontier in bridging the gap between unstructured visually rich data and precise information acquisition. Unlike traditional natural image retrieval, visual documents exhibit unique characteristics defined by dense textual content, intricate layouts, and fine-grained semantic dependencies. This paper surveys the VDR landscape as a retrieval problem in its own right, rather than as the front end of a generation pipeline, and does so specifically through the lens of the Multimodal Large Language Model (MLLM) era. We begin by examining the benchmark landscape, including the recent turn toward reasoning-intensive evaluation, and then dive into the methodological evolution along two orthogonal axes: what a retriever is, spanning multimodal embedding models and reranker models, and how it is deployed, from single-stage retrieval through Retrieval-Augmented Generation (RAG) to Agentic systems. Cutting across both, we analyse the representation--efficiency trade-off that late interaction imposes. We ground these categories in leaderboard evidence on where the empirical frontier actually lies, and close by identifying persistent challenges and outlining promising future directions for multimodal document intelligence.