mlsanityData to 14 Oct 2026

CoRe-MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG

arXiv 2506.02544 ↗First posted 3 Jun 2025cs.CL cs.AI cs.IRcited by 18

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

also in 1 more
Abstract (arXiv)

Multimodal Retrieval-Augmented Generation (MMRAG) has been introduced to enhance Multimodal Large Language Models by incorporating externally retrieved multimodal knowledge, but it introduces two challenges: Parametric-Retrieved Knowledge Inconsistency (PRKI), where discrepancies between parametric and retrieved knowledge create uncertainty in determining reliability, and Visual-Textual Knowledge Inconsistency (VTKI), where misalignment between visual and textual sources disrupts entity representation. To address these challenges, we propose Cross-source knowledge Reconciliation for Multimodal RAG (CoRe-MMRAG), a novel end-to-end framework that effectively reconciles inconsistencies across knowledge sources. CoRe-MMRAG follows a four-stage pipeline: it first generates an internal response from parametric knowledge, then selects the most relevant multimodal evidence via joint similarity assessment, generates an external response, and finally integrates both to produce a reliable answer. Additionally, a specialized training paradigm enhances knowledge source discrimination, multimodal integration, and unified answer generation. Experiments on KB-VQA benchmarks show that CoRe-MMRAG achieves substantial improvements over baseline methods, achieving 5.6% and 9.3% performance gains on InfoSeek and Encyclopedic-VQA, respectively.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 26 of its 36 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

26 of 36 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024EchoSight: Advancing Visual-Language Models with Wiki Knowledge17 Jul
  2. 2024Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMs23 Apr

Other references · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  9. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  10. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  11. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  12. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
12 more
  1. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  2. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  3. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  4. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  5. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  6. 2019PlotQA: Reasoning over Scientific Plotscited by 321
  7. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  8. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  9. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 107
  10. 2024MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Modelscited by 42
  11. 2021Interest-aware Message-Passing GCN for Recommendationcited by 39
  12. 2024Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understandingcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 13.

18 papers.

Sort:

By area:

  1. 2025OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrievalcited by 40
  2. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  3. 2026Air-Know: Arbiter-Calibrated Knowledge-Internalizing Robust Network for Composed Image Retrievalcited by 18
  4. 2026MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Networkcited by 17
  5. 2026ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrievalcited by 17
  6. 2026TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrievalcited by 15
  7. 2026COMBINER: Composed Image Retrieval Guided by Attribute-based Neighbor Relationscited by 11
  8. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  9. 2026R3G: A Reasoning-Retrieval-Reranking Framework for Vision-Centric Answer Generationcited by 4
  10. 2026CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answeringcited by 2
  11. 2025CMRAG: Co-modality-based visual document retrieval and question answeringcited by 1
  12. 2025M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAGcited by 1
6 more
  1. 2026IMAGINE: Adaptive Schema-Imagery Enhanced Composition for Composed Video Retrievalcited by 1
  2. 2026Ground Then Rank: Revisiting Knowledge-Based VQA with Training-Free Entity Identificationcited by 1
  3. 2026Identifying and Resolving Pitfalls of Knowledge-Based VQA Benchmarks: Auditing, Repairing, and Augmentingcited by 0
  4. 2026Reason Before You Retrieve: Agentic Planning for Multi-modal RAGcited by 0
  5. 2026DualG-MRAG: Decoupling Macro-Reasoning and Micro-Matching for Multimodal Retrieval-Augmented Generationcited by 0
  6. 2026CrossModalQA: A Cross-modal and Multi-hop Benchmark for Multimodal Retrieval-augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation
  2. 2026MKG-RAG-Bench: Benchmarking Retrieval in Multimodal Knowledge Graph-Augmented Generation
  3. 2025RAG-Anything: All-in-One RAG Framework
  4. 2025UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG
  5. 2025MegaRAG: Multimodal Knowledge Graph-Based Retrieval Augmented Generation
  6. 2025OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrieval
  7. 2025mKG-RAG: Leveraging Multimodal Knowledge Graphs in Retrieval-Augmented Generation for Knowledge-intensive VQA
  8. 2026CLIMB: Confidence-Guided Complementary Evidence for Multimodal Retrieval-Augmented Generation