mlsanityData to 14 Oct 2026

Evaluating VisualRAG: Quantifying Cross-Modal Performance in Enterprise Document Understanding

arXiv 2506.21604 ↗First posted 19 Jun 2025cs.IR cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Current evaluation frameworks for multimodal generative AI struggle to establish trustworthiness, hindering enterprise adoption where reliability is paramount. We introduce a systematic, quantitative benchmarking framework to measure the trustworthiness of progressively integrating cross-modal inputs such as text, images, captions, and OCR within VisualRAG systems for enterprise document intelligence. Our approach establishes quantitative relationships between technical metrics and user-centric trust measures. Evaluation reveals that optimal modality weighting with weights of 30% text, 15% image, 25% caption, and 30% OCR improves performance by 57.3% over text-only baselines while maintaining computational efficiency. We provide comparative assessments of foundation models, demonstrating their differential impact on trustworthiness in caption generation and OCR extraction-a vital consideration for reliable enterprise AI. This work advances responsible AI deployment by providing a rigorous framework for quantifying and enhancing trustworthiness in multimodal RAG for critical enterprise applications.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 33 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

18 of 33 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  6. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  7. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  8. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
  9. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  10. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  11. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  12. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
6 more
  1. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
  2. 2021TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene textcited by 171
  3. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  4. 2024TableRAG: Million-Token Table Understanding with Language Modelscited by 41
  5. 2020Document Visual Question Answering Challenge 2020cited by 20
  6. 2025Visual RAG: Expanding MLLM visual knowledge without fine-tuningcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Benchmarking Complex Multimodal Document Processing Pipelines: A Unified Evaluation Framework for Enterprise AI
  2. 2025Building Trustworthy Multimodal AI: A Review of Fairness, Transparency, and Ethics in Vision-Language Tasks
  3. 2025Trust in Vision-Language Models: Insights from a Participatory User Workshop
  4. 2026ParseBench: A Document Parsing Benchmark for AI Agents
  5. 2026Evaluating the Evaluators: Diagnosing Large Multimodal Models for AI-Generated Image Assessment
  6. 2026Beyond Accuracy: Robustness, Cost, and Governance Trade-offs for Vision-Language Models in Templated Document Extraction
  7. 2025Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospects
  8. 2025Emerging Properties in Unified Multimodal Pretraining