mlsanityData to 14 Oct 2026

ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents

arXiv 2502.18017 ↗First posted 25 Feb 2025cs.CV cs.AI cs.CLcited by 66

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Understanding information from visually rich documents remains a significant challenge for traditional Retrieval-Augmented Generation (RAG) methods. Existing benchmarks predominantly focus on image-based question answering (QA), overlooking the fundamental challenges of efficient retrieval, comprehension, and reasoning within dense visual documents. To bridge this gap, we introduce ViDoSeek, a novel dataset designed to evaluate RAG performance on visually rich documents requiring complex reasoning. Based on it, we identify key limitations in current RAG approaches: (i) purely visual retrieval methods struggle to effectively integrate both textual and visual features, and (ii) previous approaches often allocate insufficient reasoning tokens, limiting their effectiveness. To address these challenges, we propose ViDoRAG, a novel multi-agent RAG framework tailored for complex reasoning across visual documents. ViDoRAG employs a Gaussian Mixture Model (GMM)-based hybrid strategy to effectively handle multi-modal retrieval. To further elicit the model's reasoning capabilities, we introduce an iterative agent workflow incorporating exploration, summarization, and reflection, providing a framework for investigating test-time scaling in RAG domains. Extensive experiments on ViDoSeek validate the effectiveness and generalization of our approach. Notably, ViDoRAG outperforms existing methods by over 10% on the competitive ViDoSeek benchmark. The code is available at https://github.com/Alibaba-NLP/ViDoRAG.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 21 of its 35 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

21 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents14 Oct

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  6. 2015VQA: Visual Question Answeringcited by 2,506
  7. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  8. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  9. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  10. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  11. 2021MetaICL: Learning to Learn In Contextcited by 454
  12. 2019PlotQA: Reasoning over Scientific Plotscited by 321
8 more
  1. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  2. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 179
  3. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 122
  4. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 108
  5. 2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 66
  6. 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 57
  7. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 56
  8. 2024In Defense of RAG in the Era of Long-Context Language Modelscited by 53

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 28; 2026: 38.

66 papers.

Sort:

By area:

  1. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 149
  2. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 116
  3. 2025VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learningcited by 68
  4. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
  5. 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agentscited by 42
  6. 2026ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarioscited by 26
  7. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  8. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 22
  9. 2025UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAGcited by 21
  10. 2025AttentionRAG: Attention-Guided Context Pruning in Retrieval-Augmented Generationcited by 19
  11. 2025Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understandingcited by 18
  12. 2026GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughtscited by 17
38 more
  1. 2025DocLens : A Tool-Augmented Multi-Agent Framework for Long Visual Document Understandingcited by 15
  2. 2025LastingBench: Defend Benchmarks Against Knowledge Leakagecited by 14
  3. 2025Patho-AgenticRAG: Towards Multimodal Agentic Retrieval-Augmented Generation for Pathology VLMs via Reinforcement Learningcited by 13
  4. 2025DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Actioncited by 12
  5. 2025Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Modelscited by 11
  6. 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 10
  7. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  8. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  9. 2025Zero-shot Multimodal Document Retrieval via Cross-modal Question Generationcited by 6
  10. 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?cited by 5
  11. 2025HKRAG: Holistic Knowledge Retrieval-Augmented Generation Over Visually-Rich Documentscited by 5
  12. 2026VimRAG: Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graphcited by 5
  13. 2026AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generationcited by 5
  14. 2026Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representationscited by 4
  15. 2025SlideAgent: Hierarchical Agentic Framework for Multi-Page Visual Document Understandingcited by 3
  16. 2025Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agentscited by 3
  17. 2025CogDoc: Towards Unified thinking in Documentscited by 3
  18. 2026Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMscited by 3
  19. 2026Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchainscited by 3
  20. 2025Attention Basin: Why Contextual Position Matters in Large Language Modelscited by 2
  21. 2025TextlessRAG: End-to-End Visual Document RAG by Speech Without Textcited by 2
  22. 2025ALDEN: Reinforcement Learning for Active Navigation and Evidence Gathering in Long Documentscited by 2
  23. 2025CMRAG: Co-modality-based visual document retrieval and question answeringcited by 1
  24. 2025M3Prune: Hierarchical Communication Graph Pruning for Efficient Multi-Modal Multi-Agent Retrieval-Augmented Generationcited by 1
  25. 2026BayesRAG: Probabilistic Mutual Evidence Corroboration for Multimodal Retrieval-Augmented Generationcited by 1
  26. 2026G2-Reader: Dual Evolving Graphs for Multimodal Document QAcited by 1
  27. 2026FLOWREADER: Min-Cost Flow Optimization for Multi-Modal Long Document Q&Acited by 1
  28. 2026VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinementcited by 1
  29. 2026HVM-GraphRAG: Holistic-View Multimodal Graph Retrieval-Augmented Generation on Complex Documentcited by 1
  30. 2026AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrievalcited by 1
  31. 2025ADSeeker: A Knowledge-Grounded Reasoning Framework for Industry Anomaly Detection and Reasoningcited by 0
  32. 2026Index Light, Reason Deep: Deferred Visual Ingestion for Visual-Dense Document Question Answeringcited by 0
  33. 2026Evo-Retriever: LLM-Guided Curriculum Evolution with Viewpoint-Pathway Collaboration for Multimodal Document Retrievalcited by 0
  34. 2026LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrievalcited by 0
  35. 2026UniDoc-RL: Coarse-to-Fine Visual RAG with Hierarchical Actions and Dense Rewardscited by 0
  36. 2026ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstructioncited by 0
  37. 2026LFRAG: Layout-oriented Fine-grained Retrieval-Augmented Generation on Multimodal Document Understandingcited by 0
  38. 2026Test-Time Scaling in Multimodal Foundation Models: A Comprehensive Survey of Generation and Reasoningcited by 0

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents
  2. 2026KIRA: Knowledge-Intensive Image Retrieval and Reasoning Architecture for Specialized Visual Domains
  3. 2025QA-Dragon: Query-Aware Dynamic RAG System for Knowledge-Intensive Visual Question Answering
  4. 2025Visual-RAG: Benchmarking Text-to-Image Retrieval Augmented Generation for Visual Knowledge Intensive Queries
  5. 2026Learning to Search: A Decision-Based Agent for Knowledge-Based Visual Question Answering
  6. 2025LAD-RAG: Layout-aware Dynamic RAG for Visually-Rich Document Understanding
  7. 2026Doc-V*:Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA
  8. 2025Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Engines