mlsanityData to 7 Oct 2026

ColGraphRAG: Late-Interaction Evidence Retrieval for Multimodal GraphRAG

arXiv 2607.16208 ↗First posted 9 May 2026cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Graph-grounded multimodal question answering organizes text, tables, and images in a structured evidence graph, yet end-to-end accuracy depends on which multimodal assets are ranked highly enough to enter downstream reasoning; for graph-linked images, single-vector bi-encoder similarity can discard patch- and token-level structure needed for fine-grained alignment. We evaluate replacing the visual candidate-ranking operator over graph-linked image nodes with late-interaction MaxSim-style multi-vector scoring in the ColBERT/ColPali lineage, while keeping offline graph construction, text- and table-side retrieval, structured extraction, and downstream reasoning unchanged. On MultimodalQA, this change is associated with improved retrieval-stage point estimates for graph-linked image candidates and downstream QA gains, with larger movement where visual evidence matters most and mixed trends on text-dominant questions; we interpret the pattern as mechanism-level evidence for graph-linked visual evidence inclusion, while broader validation and finer graph-level diagnostics remain important future work.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 40 references that are in the map. First posted on arXiv 9 May 2026.

Builds on

36 of 40 references.

References · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  6. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  7. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  8. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  9. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  10. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  11. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  12. 2022Binding Language Models in Symbolic Languagescited by 224
24 more
  1. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 216
  2. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 190
  3. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  4. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  5. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  6. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 72
  7. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 70
  8. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documentscited by 57
  9. 2023Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answeringcited by 51
  10. 2025MMDocIR: Benchmarking Multimodal Retrieval for Long Documentscited by 43
  11. 2024PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrieverscited by 42
  12. 2024VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generationcited by 40
  13. 2024M-Longdoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Frameworkcited by 36
  14. 2025REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmarkcited by 34
  15. 2022Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generationcited by 28
  16. 2025SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinementcited by 27
  17. 2025Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Modelcited by 24
  18. 2024GRAM: Global Reasoning for Multi-Page VQAcited by 23
  19. 2026ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarioscited by 22
  20. 2023MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Imagescited by 18
  21. 2023Multimodal Graph Transformer for Multimodal Question Answeringcited by 10
  22. 2023MoqaGPT : Zero-Shot Multi-modal Open-domain Question Answering with Large Language Modelcited by 10
  23. 2025SuperRAG: Beyond RAG with Layout-Aware Graph Modelingcited by 9
  24. 2026Nemotron ColEmbed V2: Top-Performing Late Interaction Embedding Models for Visual Document Retrievalcited by 8

Built on by

Not cited yet.