mlsanityData to 14 Oct 2026

Evaluating Modern RAG: Textual, Multimodal, Dense, and Late Interaction Pipelines

arXiv 2608.23176 ↗First posted 24 Aug 2026cs.IRcited by 0

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Retrieval-augmented generation (RAG) systems have traditionally relied on text-based pipelines that extract and retrieve information from documents. While efficient and lightweight, these approaches often struggle with documents where meaning is conveyed through layout, tables, and visual elements. Recent advances in multimodal pipelines, powered by vision-language models (VLMs), improve retrieval quality by jointly encoding visual and textual signals, but at increased computational and memory cost. We propose a quantitative, data-driven selection methodology that guides practitioners in choosing the most appropriate RAG pipeline for a given document corpus based on empirical effectiveness and resource constraints. We evaluate contemporary textual and multimodal pipelines, including dense and late-interaction architectures, analyze their trade-offs, and provide actionable guidance for balancing retrieval performance with system efficiency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 14 of its 19 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

14 of 19 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation3 Dec
  2. 2024Beyond Text: Optimizing RAG with Multimodal Inputs for Industrial Applications29 Oct
  3. 2024Financial Report Chunking for Effective Retrieval Augmented Generation5 Feb
  4. 2022Retrieval Augmented Visual Question Answering with Outside Knowledge7 Oct
  5. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text6 Oct

Other references · 9

  1. 2024Qwen2.5 Technical Reportcited by 4,942
  2. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  3. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 195
  4. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
  5. 2024LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domaincited by 118
  6. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 110
  7. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 85
  8. 2024Improving Retrieval for RAG based Question Answering Models on Financial Documentscited by 49
  9. 2025REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmarkcited by 35

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?
  2. 2024Searching for Best Practices in Retrieval-Augmented Generation
  3. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents
  4. 2025Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Survey
  5. 2026Retrieval or Representation? Reassessing Benchmark Gaps in Multilingual and Visually Rich RAG
  6. 2026ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarios
  7. 2025FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation
  8. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Models