mlsanityData to 7 Oct 2026

VISPATH: Visual-Intent-Guided Path Reasoning for Multimodal Knowledge Graph Question Answering

arXiv 2609.20843 ↗First posted 4 Aug 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Knowledge graph question answering (KGQA) enables models to answer natural-language questions through structured graph reasoning and has achieved substantial progress across many benchmarks and applications. Recently, multimodal KGQA (MM-KGQA) has attracted increasing attention because many questions require jointly using multimodal inputs and KG evidence. However, existing MM-KGQA methods typically use multimodal information only for starting entity grounding or evidence retrieval, after which multi-hop reasoning degenerates into text-only graph search. As a result, they cannot exploit multimodal cues that become important at intermediate hops. To address this limitation, we propose VISPATH, a visual-intent-guided path reasoning framework for MM-KGQA. VISPATH first identifies a reliable starting entity by combining multimodal grounding with graph-structural cues. It then performs intent-guided path discovery by recomputing hop-specific multimodal intent from the input, question, and current partial paths, so that each expansion is guided by the current reasoning state. The discovered paths are further refined through reasoning-chain pruning, which evaluates candidate paths as complete evidence chains based on their consistency with the question, reasoning sketch, and hop-specific intent. Finally, VISPATH checks whether the selected evidence is sufficient for answer generation. We further construct VISPATH-Bench, a benchmark for evaluating multimodal multi-hop reasoning over KGs, covering questions that require two to four hops over KG paths. Extensive experiments on VISPATH-Bench and three additional multimodal QA benchmarks show that VISPATH consistently outperforms strong baselines. Notably, with GPT-4o as the backbone, VISPATH surpasses GPT-5.4 on VISPATH-Bench, achieving a 10.6% relative improvement in average accuracy and a 13.1% improvement at 2-hop reasoning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 58 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

39 of 58 references.

References · 39

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  6. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  7. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  8. 2020Knowledge Graphscited by 619
  9. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  10. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  11. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  12. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
27 more
  1. 2024LightRAG: Simple and Fast Retrieval-Augmented Generationcited by 310
  2. 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
  3. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  4. 2019PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Textcited by 182
  5. 2018Open Domain Question Answering Using Early Fusion of Knowledge Bases and Textcited by 167
  6. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 121
  7. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 120
  8. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 105
  9. 2019MMKG: Multi-Modal Knowledge Graphscited by 104
  10. 2021Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signalscited by 100
  11. 2021RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answeringcited by 86
  12. 2024Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphscited by 77
  13. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  14. 2023MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Modelscited by 73
  15. 2016Image-embodied Knowledge Representation Learningcited by 68
  16. 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
  17. 2024Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generationcited by 63
  18. 2021TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graphcited by 60
  19. 2024Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoningcited by 44
  20. 2020VisualSem: A High-quality Knowledge Graph for Vision and Languagecited by 30
  21. 2023UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Trainingcited by 14
  22. 2025mKG-RAG: Leveraging Multimodal Knowledge Graphs in Retrieval-Augmented Generation for Knowledge-intensive VQAcited by 10
  23. 2025ReasonVQA: A Multi-hop Reasoning Benchmark with Structural Knowledge for Visual Question Answeringcited by 7
  24. 2023AspectMMKG: A Multi-modal Knowledge Graph with Aspect-aware Entitiescited by 5
  25. 2023M^2ConceptBase: A Fine-Grained Aligned Concept-Centric Multimodal Knowledge Basecited by 2
  26. 2025VAT-KG: Knowledge-Intensive Multimodal Knowledge Graph Dataset for Retrieval-Augmented Generationcited by 2
  27. 2025MMhops-R1: Multimodal Multi-hop Reasoningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Theseus in the Graph: Towards Traceable Multi-Hop Graph Navigation
  2. 2025BYOKG-RAG: Multi-Strategy Graph Retrieval for Knowledge Graph Question Answering
  3. 2025StepChain GraphRAG: Reasoning Over Knowledge Graphs for Multi-Hop Question Answering
  4. 2026M3-VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering
  5. 2025DAMR: Efficient and Adaptive Context-Aware Knowledge Graph Question Answering with LLM-Guided MCTS
  6. 2025Multimodal Multihop Source Retrieval for Web Question Answering
  7. 2026S-Path-RAG: Semantic-Aware Shortest-Path Retrieval Augmented Generation for Multi-Hop Knowledge Graph Question Answering
  8. 2025Search-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphs