mlsanityData to 7 Oct 2026

Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding

arXiv 2510.15253 ↗First posted 17 Oct 2025cs.CL cs.CVcited by 9

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Document understanding is critical for applications from financial analysis to scientific discovery. Current approaches, whether OCR-based pipelines feeding Large Language Models (LLMs) or native Multimodal LLMs (MLLMs), face key limitations: the former loses structural detail, while the latter struggles with context modeling. Retrieval-Augmented Generation (RAG) helps ground models in external data, but documents' multimodal nature, i.e., combining text, tables, charts, and layout, demands a more advanced paradigm: Multimodal RAG. This approach enables holistic retrieval and reasoning across all modalities, unlocking comprehensive document intelligence. Recognizing its importance, this paper presents a systematic survey of Multimodal RAG for document understanding. We propose a taxonomy based on domain, retrieval modality, and granularity, and review advances involving graph structures and agentic frameworks. We also summarize key datasets, benchmarks, applications and industry deployment, and highlight open challenges in efficiency, fine-grained representation, and robustness, providing a roadmap for future progress in document AI.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 99 of its 181 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

99 of 181 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG4 Oct
  2. 2025Enhancing Document VQA Models via Retrieval-Augmented Generation26 Aug
  3. 2025M2IO-R1: An Efficient RL-Enhanced Reasoning Framework for Multimodal Retrieval Augmented Multimodal Generation8 Aug
  4. 2025mKG-RAG: Leveraging Multimodal Knowledge Graphs in Retrieval-Augmented Generation for Knowledge-intensive VQA7 Aug
  5. 2025Vision-Guided Chunking Is All You Need: Enhancing RAG with Multimodal Document Understanding19 Jun
  6. 2025CoRe-MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG3 Jun
  7. 2025VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning28 May
  8. 2025A Multi-Granularity Retrieval Framework for Visually-Rich Documents1 May
  9. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents14 Apr
  10. 2025CollEX -- A Multimodal Agentic RAG System Enabling Interactive Exploration of Scientific Collections10 Apr
  11. 2025A Survey of Multimodal Retrieval-Augmented Generation26 Mar
  12. 2025ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents25 Feb
4 more
  1. 2024VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation14 Dec
  2. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding7 Nov
  3. 2024Beyond Text: Optimizing RAG with Multimodal Inputs for Industrial Applications29 Oct
  4. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents14 Oct

Other references · 83

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  7. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  8. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  9. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  10. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  11. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  12. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 519
38 more
  1. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  2. 2020Understanding the Behaviour of Contrastive Losscited by 397
  3. 2019Scene Text Visual Question Answeringcited by 327
  4. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  5. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  6. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  7. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  8. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 242
  9. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  10. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decodingcited by 198
  11. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
  12. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  13. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  14. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  15. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113
  16. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 107
  17. 2022Unified Pretraining Framework for Document Understandingcited by 85
  18. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  19. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
  20. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  21. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
  22. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 70
  23. 2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systemscited by 69
  24. 2025MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understandingcited by 66
  25. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  26. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbationscited by 64
  27. 2024Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generationcited by 63
  28. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 59
  29. 2020FQuAD: French Question Answering Datasetcited by 58
  30. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 51
  31. 2024MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generationcited by 43
  32. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 42
  33. 2024UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysiscited by 40
  34. 2025MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactioncited by 36
  35. 2024Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectorycited by 35
  36. 2024AI Hallucinations: A Misnomer Worth Clarifyingcited by 34
  37. 2025LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarkscited by 34
  38. 2020A Survey of Deep Learning Approaches for OCR and Document Understandingcited by 32

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 8
  2. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  3. 2026CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embeddingcited by 6
  4. 2026MINER: Mining Multimodal Internal Representation for Efficient Retrievalcited by 2
  5. 2026Very Efficient Listwise Multimodal Reranking for Long Documentscited by 1
  6. 2026MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&Acited by 1
  7. 2026FLOWREADER: Min-Cost Flow Optimization for Multi-Modal Long Document Q&Acited by 1
  8. 2026GVD: Governed Versioning and Deduplication for Document Repositoriescited by 0
  9. 2026Amnesia by Design, Memory By Necessity: Persistent State for Document Intelligencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RAG-Anything: All-in-One RAG Framework
  2. 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?
  3. 2025Zero-Shot Document Understanding using Pseudo Table of Contents-Guided Retrieval-Augmented Generation
  4. 2026Hierarchical Evidence-Driven Reasoning for Long Document Understanding
  5. 2026Evaluating Modern RAG: Textual, Multimodal, Dense, and Late Interaction Pipelines
  6. 2025Multimodal RAG for Unstructured Data:Leveraging Modality-Aware Knowledge Graphs with Hybrid Retrieval
  7. 2025MegaRAG: Multimodal Knowledge Graph-Based Retrieval Augmented Generation
  8. 2026LFRAG: Layout-oriented Fine-grained Retrieval-Augmented Generation on Multimodal Document Understanding