mlsanityData to 7 Oct 2026

Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval

arXiv 2602.19961 ↗First posted 23 Feb 2026cs.CL cs.IRcited by 8

Not on the map. Why?

Abstract (arXiv)

With the rapid proliferation of multimodal information, Visual Document Retrieval (VDR) has emerged as a critical frontier in bridging the gap between unstructured visually rich data and precise information acquisition. Unlike traditional natural image retrieval, visual documents exhibit unique characteristics defined by dense textual content, intricate layouts, and fine-grained semantic dependencies. This paper surveys the VDR landscape as a retrieval problem in its own right, rather than as the front end of a generation pipeline, and does so specifically through the lens of the Multimodal Large Language Model (MLLM) era. We begin by examining the benchmark landscape, including the recent turn toward reasoning-intensive evaluation, and then dive into the methodological evolution along two orthogonal axes: what a retriever is, spanning multimodal embedding models and reranker models, and how it is deployed, from single-stage retrieval through Retrieval-Augmented Generation (RAG) to Agentic systems. Cutting across both, we analyse the representation--efficiency trade-off that late interaction imposes. We ground these categories in leaderboard evidence on where the empirical frontier actually lies, and close by identifying persistent challenges and outlining promising future directions for multimodal document intelligence.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 180 of its 230 references that are in the map. First posted on arXiv 23 Feb 2026.

Builds on

180 of 230 references.

References · 180

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  3. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  4. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  5. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 242
  6. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  7. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  8. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  9. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 201
  10. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
  11. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 165
  12. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
38 more
  1. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 153
  2. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  3. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 136
  4. 2022Revisiting Neural Scaling Laws in Language and Visioncited by 129
  5. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 128
  6. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 120
  7. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 110
  8. 2024Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Expertscited by 107
  9. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 107
  10. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  11. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  12. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 89
  13. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
  14. 2024MegaPairs: Massive Data Synthesis For Universal Multimodal Retrievalcited by 84
  15. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  16. 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
  17. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 72
  18. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 70
  19. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 69
  20. 2025Rank1: Test-Time Compute for Reranking in Information Retrievalcited by 68
  21. 2025VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learningcited by 68
  22. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  23. 2024VISTA: Visualized Text Embedding For Universal Multi-Modal Retrievalcited by 67
  24. 2025MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understandingcited by 66
  25. 2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agentscited by 66
  26. 2025ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agentscited by 64
  27. 2025Deep Research: A Survey of Autonomous Research Agentscited by 63
  28. 2021Document AI: Benchmarks, Models and Applicationscited by 61
  29. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 59
  30. 2025ViDoRe Benchmark V2: Raising the Bar for Visual Retrievalcited by 58
  31. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documentscited by 57
  32. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  33. 2025LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learningcited by 54
  34. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmarkcited by 53
  35. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 52
  36. 2024MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Trainingcited by 50
  37. 2025mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Datacited by 49
  38. 2024A Closer Look into Mixture-of-Experts in Large Language Modelscited by 47

50 most cited of 180 All 180 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Multi-Granularity Retrieval Framework for Visually-Rich Documents
  2. 2026MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval
  3. 2025Recurrence-Enhanced Vision-and-Language Transformers for Robust Multimodal Document Retrieval
  4. 2025Reproducibility, Replicability, and Insights into Visual Document Retrieval with Late Interaction
  5. 2024Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents
  6. 2026HIVE: Query, Hypothesize, Verify An LLM Framework for Multimodal Reasoning-Intensive Retrieval
  7. 2026Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval
  8. 2026DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding