Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering
On the map: LLM reasoning → Knowledge- and tool-augmented reasoning
Abstract (arXiv)
Large language models (LLMs) have shown remarkable capabilities in natural language processing. However, in knowledge graph question answering tasks (KGQA), there remains the issue of answering questions that require multi-hop reasoning. Existing methods rely on entity vector matching, but the purpose of the question is abstract and difficult to match with specific entities. As a result, it is difficult to establish reasoning paths to the purpose, which leads to information loss and redundancy. To address this issue, inspired by human reverse thinking, we propose Ontology-Guided Reverse Thinking (ORT), a novel framework that constructs reasoning paths from purposes back to conditions. ORT operates in three key phases: (1) using LLM to extract purpose labels and condition labels, (2) constructing label reasoning paths based on the KG ontology, and (3) using the label reasoning paths to guide knowledge retrieval. Experiments on the WebQSP and CWQ datasets show that ORT achieves state-of-the-art performance and significantly enhances the capability of LLMs for KGQA.
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 43 references that are in the map. First posted on arXiv 17 Feb 2025.
Builds on
23 of 43 references.
Closest ancestors · 3
Same direction, newest first
- 2024Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning18 Oct
- 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning2 Oct
- 2022UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph2 Dec
Other references · 20
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
- 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
- 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
- 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 238
- 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
- 2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
- 2018Open Domain Question Answering Using Early Fusion of Knowledge Bases and Textcited by 167
- 2021Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signalscited by 100
- 2023Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answeringcited by 97
- 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 95
- 2022Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answeringcited by 93
8 more
- 2022DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Basescited by 83
- 2023Text Summarization Using Large Language Models: A Comparative Study of MPT-7b-instruct, Falcon-7b-instruct, and OpenAI Chat-GPT Modelscited by 35
- 2024Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphscited by 24
- 2024GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translatorscited by 22
- 2024TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Treescited by 19
- 2024KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledgecited by 16
- 2024HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMscited by 16
- 2024BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solvingcited by 15
Built on by
Citing papers per year: 2025: 7; 2026: 8.
15 papers.
Sort:
By area:
- 2025Align-GRAG: Anchor and Rationale Guided Dual Alignment for Graph Retrieval-Augmented Generationcited by 9
- 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8
- 2025Search-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphscited by 5
- 2025PRoH: Dynamic Planning and Reasoning over Knowledge Hypergraphs for Retrieval-Augmented Generationcited by 2
- 2026Autonomous Knowledge Graph Exploration with Adaptive Breadth-Depth Retrievalcited by 2
- 2026Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learningcited by 2
- 2026FedV-KGQA: Multi-Hop Question Answering over Vertically Partitioned Knowledge Graphscited by 2
- 2025TextMineX: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Actioncited by 0
- 2025NOEM3A: a Neuro-symbolic Ontology-Enhanced Method for Multi-intent understanding in Mobile Agentscited by 0
- 2025StruProKGR: A Structural and Probabilistic Framework for Sparse Knowledge Graph Reasoningcited by 0
- 2026Schema-Aware Planning and Hybrid Knowledge Toolset for Reliable Knowledge Graph Triple Verificationcited by 0
- 2026MAB-DQA: Addressing Query Aspect Importance in Document Question Answering with Multi-Armed Banditscited by 0
3 more
- 2026When Correct Edges Cannot Be Verified: A Provenance Gap in Incomplete KGQA and a Provenance-Favoring Completion Policycited by 0
- 2026Ontology-Guided Evidence Path Inference for Multi-hop Knowledge Graph Question Answeringcited by 0
- 2026A Multi-Agent Framework for Zero-Dimensional Reduced-Order Model Planningcited by 0
- 2026FedV-KGQA: Multi-Hop Question Answering over Vertically Partitioned Knowledge Graphs25 Aug 2026
- 2026Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learning20 Jul 2026
- 2026A Multi-Agent Framework for Zero-Dimensional Reduced-Order Model Planning13 Jul 2026
- 2026Ontology-Guided Evidence Path Inference for Multi-hop Knowledge Graph Question Answering26 Jun 2026
- 2026When Correct Edges Cannot Be Verified: A Provenance Gap in Incomplete KGQA and a Provenance-Favoring Completion Policy14 Jun 2026
- 2026MAB-DQA: Addressing Query Aspect Importance in Document Question Answering with Multi-Armed Bandits10 Apr 2026
- 2026Schema-Aware Planning and Hybrid Knowledge Toolset for Reliable Knowledge Graph Triple Verification5 Apr 2026
- 2026Autonomous Knowledge Graph Exploration with Adaptive Breadth-Depth Retrieval20 Jan 2026
- 2025StruProKGR: A Structural and Probabilistic Framework for Sparse Knowledge Graph Reasoning14 Dec 2025
- 2025NOEM3A: a Neuro-symbolic Ontology-Enhanced Method for Multi-intent understanding in Mobile Agents24 Nov 2025
- 2025PRoH: Dynamic Planning and Reasoning over Knowledge Hypergraphs for Retrieval-Augmented Generation14 Oct 2025
- 2025Search-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphs9 Oct 2025
3 more
- 2025TextMineX: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Action18 Sep 2025
- 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Models12 Sep 2025
- 2025Align-GRAG: Anchor and Rationale Guided Dual Alignment for Graph Retrieval-Augmented Generation22 May 2025
- 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8
- 2026Autonomous Knowledge Graph Exploration with Adaptive Breadth-Depth Retrievalcited by 2
- 2026Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learningcited by 2
- 2026FedV-KGQA: Multi-Hop Question Answering over Vertically Partitioned Knowledge Graphscited by 2
- 2025TextMineX: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Actioncited by 0
- 2025StruProKGR: A Structural and Probabilistic Framework for Sparse Knowledge Graph Reasoningcited by 0
- 2026Schema-Aware Planning and Hybrid Knowledge Toolset for Reliable Knowledge Graph Triple Verificationcited by 0
- 2026MAB-DQA: Addressing Query Aspect Importance in Document Question Answering with Multi-Armed Banditscited by 0
- 2026When Correct Edges Cannot Be Verified: A Provenance Gap in Incomplete KGQA and a Provenance-Favoring Completion Policycited by 0
- 2026Ontology-Guided Evidence Path Inference for Multi-hop Knowledge Graph Question Answeringcited by 0
- 2026FedV-KGQA: Multi-Hop Question Answering over Vertically Partitioned Knowledge Graphs25 Aug 2026
- 2026Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learning20 Jul 2026
- 2026Ontology-Guided Evidence Path Inference for Multi-hop Knowledge Graph Question Answering26 Jun 2026
- 2026When Correct Edges Cannot Be Verified: A Provenance Gap in Incomplete KGQA and a Provenance-Favoring Completion Policy14 Jun 2026
- 2026MAB-DQA: Addressing Query Aspect Importance in Document Question Answering with Multi-Armed Bandits10 Apr 2026
- 2026Schema-Aware Planning and Hybrid Knowledge Toolset for Reliable Knowledge Graph Triple Verification5 Apr 2026
- 2026Autonomous Knowledge Graph Exploration with Adaptive Breadth-Depth Retrieval20 Jan 2026
- 2025StruProKGR: A Structural and Probabilistic Framework for Sparse Knowledge Graph Reasoning14 Dec 2025
- 2025TextMineX: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Action18 Sep 2025
- 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Models12 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025RJE: A Retrieval-Judgment-Exploration Framework for Efficient Knowledge Graph Question Answering with LLMs
- 2025iQUEST: An Iterative Question-Guided Framework for Knowledge Base Question Answering
- 2025Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering
- 2025The Role of Exploration Modules in Small Language Models for Knowledge Graph Question Answering
- 2025KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration
- 2024Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation
- 2025Fusing Knowledge and Language: A Comparative Study of Knowledge Graph-Based Question Answering with LLMs
- 2025From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs