mlsanityData to 7 Oct 2026

Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation

arXiv 2608.30163 ↗First posted 31 Aug 2026cs.IR cs.CVcited by 0

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering. However, existing multimodal RAG models are primarily designed for single-image or closed-document settings and exhibit limited accuracy in realistic multi-image scenarios. Moreover, processing numerous retrieved images incurs substantial computational overhead from irrelevant visual tokens. To address these challenges, we introduce DocLongRAG, a large-scale dataset of 343K question--answer pairs, each associated with an average of 37.4 retrieved images to reflect authentic RAG workflows. Building on this dataset, we propose Doc-REFRAG, a question-guided framework that compresses visual tokens into coarse chunks and selectively expands question-relevant ones via a lightweight RL-based selector. Experiments on six benchmarks show that Doc-REFRAG outperforms eleven strong baselines, achieving state-of-the-art accuracy with significantly lower inference latency. Our resources are available at https://github.com/Collab-Gen/Doc-REFRAG.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 35 of its 52 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

35 of 52 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents14 Apr
  2. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents14 Oct

Other references · 33

  1. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  3. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  5. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  6. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  7. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
  8. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  9. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 173
  10. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 164
  11. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  12. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
21 more
  1. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  2. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113
  3. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 112
  4. 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understandingcited by 106
  5. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?cited by 76
  6. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 72
  7. 2025ViDoRe Benchmark V2: Raising the Bar for Visual Retrievalcited by 58
  8. 2024ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsificationcited by 43
  9. 2025MMDocIR: Benchmarking Multimodal Retrieval for Long Documentscited by 43
  10. 2025What Kind of Visual Tokens Do We Need? Training-free Visual Token Pruning for Multi-modal Large Language Models from the Perspective of Graphcited by 35
  11. 2024LongReward: Improving Long-context Large Language Models with AI Feedbackcited by 34
  12. 2025FOLDER: Accelerating Multi-modal Large Language Models with Enhanced Performancecited by 31
  13. 2025MM-R5: MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrievalcited by 21
  14. 2025NExtLong: Toward Effective Long-Context Training without Long Documentscited by 19
  15. 2025TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Modelscited by 19
  16. 2025Docopilot: Improving Multimodal Models for Document-Level Understandingcited by 19
  17. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 19
  18. 2024Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documentscited by 17
  19. 2023MPMQA: Multimodal Question Answering on Product Manualscited by 10
  20. 2025Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoningcited by 7
  21. 2025Vela: Scalable Embeddings with Voice Large Language Models for Multimodal Retrievalcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answering
  2. 2025CMRAG: Co-modality-based visual document retrieval and question answering
  3. 2025Visual-RAG: Benchmarking Text-to-Image Retrieval Augmented Generation for Visual Knowledge Intensive Queries
  4. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text
  5. 2026Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion
  6. 2025MegaRAG: Multimodal Knowledge Graph-Based Retrieval Augmented Generation
  7. 2025VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understanding
  8. 2025HKRAG: Holistic Knowledge Retrieval-Augmented Generation Over Visually-Rich Documents