mlsanityData to 7 Oct 2026

ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarios

arXiv 2601.08620 ↗First posted 13 Jan 2026cs.AI cs.CVcited by 22

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) pipelines must address challenges beyond simple single-document retrieval, such as interpreting visual elements (tables, charts, images), synthesizing information across documents, and providing accurate source grounding. Existing benchmarks fail to capture this complexity, often focusing on textual data, single-document comprehension, or evaluating retrieval and generation in isolation. We introduce ViDoRe v3, a comprehensive multimodal RAG benchmark featuring multi-type queries over visually rich document corpora. It covers 10 datasets across diverse professional domains, comprising ~26,000 document pages paired with 3,099 human-verified queries, each available in 6 languages. Through 12,000 hours of human annotation effort, we provide high-quality annotations for retrieval relevance, bounding box localization, and verified reference answers. Our evaluation of state-of-the-art RAG pipelines reveals that visual retrievers outperform textual ones, late-interaction models and textual reranking substantially improve performance, and hybrid or purely visual contexts enhance answer generation quality. However, current models still struggle with non-textual elements, open-ended queries, and fine-grained visual grounding. To encourage progress in addressing these challenges, the benchmark is released under a commercially permissive license at https://hf.co/vidore.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 22 of its 38 references that are in the map. First posted on arXiv 13 Jan 2026.

Builds on

22 of 38 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG4 Oct
  2. 2025ModernVBERT: Towards Smaller Visual Document Retrievers1 Oct
  3. 2025ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents25 Feb
  4. 2024VISA: Retrieval Augmented Generation with Visual Source Attribution19 Dec
  5. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding7 Nov
  6. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents14 Oct

Other references · 16

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  3. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  4. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  5. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  6. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 253
  7. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
  8. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  9. 2024BM25S: Orders of magnitude faster lexical search via eager sparse scoringcited by 87
  10. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  11. 2025ViDoRe Benchmark V2: Raising the Bar for Visual Retrievalcited by 58
  12. 2025REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmarkcited by 34
4 more
  1. 2025Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Modelcited by 24
  2. 2025FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documentscited by 15
  3. 2025Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddingscited by 12
  4. 2025Fantastic (small) Retrievers and How to Train Them: mxbai-edge-colbert-v0 Tech Reportcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 22.

22 papers.

Sort:

By area:

  1. 2026Nemotron ColEmbed V2: Top-Performing Late Interaction Embedding Models for Visual Document Retrievalcited by 8
  2. 2026CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embeddingcited by 6
  3. 2026AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generationcited by 5
  4. 2026CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligencecited by 5
  5. 2026AEC-Bench: A Multimodal Benchmark for Agentic Systems in Architecture, Engineering, and Constructioncited by 4
  6. 2026IRPAPERS: A Visual Document Benchmark for Scientific Retrieval and Question Answeringcited by 2
  7. 2026BRIDGE: Benchmark for multi-hop Reasoning In long multimodal Documents with Grounded Evidencecited by 2
  8. 2026NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrievalcited by 2
  9. 2026MINER: Mining Multimodal Internal Representation for Efficient Retrievalcited by 2
  10. 2026LEMUR: Learned Multi-Vector Retrievalcited by 1
  11. 2026MinerU-Popo: Universal Post-Processing Model for Structured Document Parsingcited by 1
  12. 2026AMES: Approximate Multi-modal Enterprise Search via Late Interaction Retrievalcited by 0
10 more
  1. 2026LITTA: Late-Interaction and Test-Time Alignment for Visually-Grounded Multimodal Retrievalcited by 0
  2. 2026Hydra: Unifying Document Retrieval and Generation in a Single Vision-Language Modelcited by 0
  3. 2026Structured Layout Priors for Robust Out-of-Distribution Visual Document Understandingcited by 0
  4. 2026ColGraphRAG: Late-Interaction Evidence Retrieval for Multimodal GraphRAGcited by 0
  5. 2026KoVRE: Training an Efficient Embedding Model for Korean Visual Document Retrievalcited by 0
  6. 2026L"etzCross: A Cross-Lingual Page-Level Benchmark for Multimodal Retrieval over Luxembourgish Documentscited by 0
  7. 2026MCite-RL: Towards Reliable Multimodal RAG via Citation-enhanced Agentic Reinforcement Learningcited by 0
  8. 2026ViSAR: Training-Free Adaptive-k Retrieval for Visual Document Question Answeringcited by 0
  9. 2026ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transportcited by 0
  10. 2026Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?
  2. 2026Evaluating Modern RAG: Textual, Multimodal, Dense, and Late Interaction Pipelines
  3. 2026Retrieval or Representation? Reassessing Benchmark Gaps in Multilingual and Visually Rich RAG
  4. 2024Searching for Best Practices in Retrieval-Augmented Generation
  5. 2025mmRAG: A Modular Benchmark for Retrieval-Augmented Generation over Text, Tables, and Knowledge Graphs
  6. 2025Visual-RAG: Benchmarking Text-to-Image Retrieval Augmented Generation for Visual Knowledge Intensive Queries
  7. 2025Stronger Baselines for Retrieval-Augmented Generation with Long-Context Language Models
  8. 2025Enhancing Document VQA Models via Retrieval-Augmented Generation