mlsanityData to 7 Oct 2026

Trust Before Fusion: QIMG-7 and Source-Aware Resolution for Polluted Multimodal RAG

arXiv 2607.10798 ↗First posted 12 Jul 2026cs.CLcited by 0

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Multimodal retrieval-augmented generation (RAG) is often evaluated with clean evidence, yet real retrieval can return topically relevant but unreliable content: false text and misleading images from corrupted metadata, entity swaps, typographic overlays, semantic edits, adversarial patches, blends, or style transfer. We introduce QIMG-7, a controlled benchmark for multimodal retrieval pollution in multi-sentence factual QA, spanning four datasets, seven image-attack families, and 16 paired clean/polluted regimes, for 1,760 evaluation rows per method. Across four generator/gate stacks, naive multimodal fusion is brittle: in the main gpt-4o-mini stack, Full-MM support drops from 0.908 with clean text to 0.490 with polluted text, often making Parametric fallback safer than retrieval. We propose source-aware trust resolution (SATR), a training-free approach that compares Parametric, Text-only, and Full-MM candidate answers and selects among candidate answers or falls back based on source reliability. The Field-Selector variant achieves the best balanced score, 0.816, improving over Full-MM by 11.7 points and over the Cascaded Router by 2.7 points. Ablations show that, in this text-first setting, explicit text-reliability modeling is the dominant driver of these gains. Overall, in text-first factual QA with multimodal retrieval conflict, our results support selective trust rather than unconditional fusion. Artifacts are available at https://github.com/SaadElDine/Trust_Before_Fusion.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 15 of its 27 references that are in the map. First posted on arXiv 12 Jul 2026.

Builds on

15 of 27 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025RAG-Check: Evaluating Multimodal Retrieval Augmented Generation Performance7 Jan
  2. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents14 Oct
  3. 2024MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models10 Oct

Other references · 12

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  4. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  5. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  6. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  7. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  8. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
  9. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  10. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  11. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  12. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievalscited by 27

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG
  2. 2026CLIMB: Confidence-Guided Complementary Evidence for Multimodal Retrieval-Augmented Generation
  3. 2026Hidden in the Metadata: Stealth Poisoning Attacks on Multimodal Retrieval-Augmented Generation
  4. 2026MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG
  5. 2026CrossModalQA: A Cross-modal and Multi-hop Benchmark for Multimodal Retrieval-augmented Generation
  6. 2025CoRe-MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG
  7. 2026BayesRAG: Probabilistic Mutual Evidence Corroboration for Multimodal Retrieval-Augmented Generation
  8. 2026RAGStress: A controlled benchmark for evaluating retrieval-augmented generation under knowledge-base degradation