mlsanityData to 14 Oct 2026

SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinement

arXiv 2506.14035 ↗First posted 16 Jun 2025cs.CV cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Document Visual Question Answering (DocVQA) is a practical yet challenging task, which is to ask questions based on documents while referring to multiple pages and different modalities of information, e.g, images and tables. To handle multi-modality, recent methods follow a similar Retrieval Augmented Generation (RAG) pipeline, but utilize Visual Language Models (VLMs) based embedding model to embed and retrieve relevant pages as images, and generate answers with VLMs that can accept an image as input. In this paper, we introduce SimpleDoc, a lightweight yet powerful retrieval - augmented framework for DocVQA. It boosts evidence page gathering by first retrieving candidates through embedding similarity and then filtering and re-ranking these candidates based on page summaries. A single VLM-based reasoner agent repeatedly invokes this dual-cue retriever, iteratively pulling fresh pages into a working memory until the question is confidently answered. SimpleDoc outperforms previous baselines by 3.2% on average on 4 DocVQA datasets with much fewer pages retrieved. Our code is available at https://github.com/ag2ai/SimpleDoc.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 40 of its 50 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

40 of 50 references.

References · 40

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  7. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  8. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  9. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  10. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  11. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  12. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
28 more
  1. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  2. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  3. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
  4. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
  5. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 220
  6. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 184
  7. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  8. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 179
  9. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 147
  10. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 135
  11. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 111
  12. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 105
  13. 2022Unified Pretraining Framework for Document Understandingcited by 85
  14. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 72
  15. 2025MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understandingcited by 67
  16. 2024StateFlow: Enhancing LLM Task-Solving through State-Driven Workflowscited by 65
  17. 2023Large Language Model based Long-tail Query Rewriting in Taobao Searchcited by 62
  18. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documentscited by 59
  19. 2020Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR)cited by 56
  20. 2024RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedbackcited by 46
  21. 2024PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makerscited by 42
  22. 2024VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generationcited by 41
  23. 2024UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysiscited by 40
  24. 2024M-Longdoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Frameworkcited by 36
  25. 2025HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generationcited by 31
  26. 2024VISA: Retrieval Augmented Generation with Visual Source Attributioncited by 28
  27. 2022V-Doc : Visual questions answers with Documentscited by 19
  28. 2024PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Samplingcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 21.

28 papers.

Sort:

By area:

  1. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
  2. 2025DocLens : A Tool-Augmented Multi-Agent Framework for Long Visual Document Understandingcited by 15
  3. 2026According to Me: Long-Term Personalized Referential Memory QAcited by 14
  4. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agentscited by 12
  5. 2026DocDancer: Towards Agentic Document-Grounded Information Seekingcited by 12
  6. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  7. 2025A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trendscited by 8
  8. 2025LAD-RAG: Layout-aware Dynamic RAG for Visually-Rich Document Understandingcited by 7
  9. 2026AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generationcited by 5
  10. 2026CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligencecited by 5
  11. 2025Hybrid-Vector Retrieval for Visually Rich Documents: Combining Single-Vector Efficiency and Multi-Vector Accuracycited by 4
  12. 2026HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answeringcited by 3
16 more
  1. 2026LensVLM: Selective Context Expansion for Compressed Visual Representation of Textcited by 2
  2. 2026Hierarchical Evidence-Driven Reasoning for Long Document Understandingcited by 2
  3. 2026PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documentscited by 1
  4. 2026VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinementcited by 1
  5. 2026Amnesia by Design, Memory By Necessity: Persistent State for Document Intelligencecited by 1
  6. 2026Diagnosing Failure Modes of Shared-State Collaboration in Resource-Constrained Visual Agentscited by 0
  7. 2026Hybrid Retriever Evolution for Multimodal Document Reasoning Agentscited by 0
  8. 2026ColGraphRAG: Late-Interaction Evidence Retrieval for Multimodal GraphRAGcited by 0
  9. 2026XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understandingcited by 0
  10. 2026DocMemo: Dynamic Evidence Discovery via Probabilistic Memory-Guided Retrieval for Multi-Modal Document Understandingcited by 0
  11. 2026DocAtlas: Long-Document Understanding as Mutable-State Interactioncited by 0
  12. 2026What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answeringcited by 0
  13. 2026Question-Guided Evidence Acquisition for Multimodal Visual Question Answeringcited by 0
  14. 2026ViSAR: Training-Free Adaptive-k Retrieval for Visual Document Question Answeringcited by 0
  15. 2026An Empirical Study of VLM Pipelines for Long-Document QAcited by 0
  16. 2026Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QAcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Document VQA Models via Retrieval-Augmented Generation
  2. 2026Doc-V*:Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA
  3. 2025Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answering
  4. 2026Towards Self-Explainable Document Visual Question Answering with Chain-of-Explanation Predictions
  5. 2026Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering
  6. 2026Index Light, Reason Deep: Deferred Visual Ingestion for Visual-Dense Document Question Answering
  7. 2022Towards Complex Document Understanding By Discrete Reasoning
  8. 2026AVIR: Adaptive Visual In-Document Retrieval for Efficient Multi-Page Document Question Answering