# KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph

arXiv: https://arxiv.org/abs/2312.15880 · HTML: https://mlsanity.com/paper/2312.15880/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 29 of its 51 references that are in the map.
- First posted on arXiv 26 Dec 2023.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 29 of 29)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [StructGPT: A General Framework for Large Language Model to Reason over Structured Data](https://mlsanity.com/paper/2305.09645/) (2023)
- [Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.09712/) (2022)
- [Variational Reasoning for Question Answering with Knowledge Graph](https://mlsanity.com/paper/1709.04071/) (2017)
- [Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text](https://mlsanity.com/paper/1809.00782/) (2018)
- [Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning](https://mlsanity.com/paper/1711.05851/) (2017)
- [Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering](https://mlsanity.com/paper/2306.04136/) (2023)
- [UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph](https://mlsanity.com/paper/2212.00959/) (2022)
- [Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals](https://mlsanity.com/paper/2101.03737/) (2021)
- [Multi-Modal Knowledge Graph Construction and Application: A Survey](https://mlsanity.com/paper/2202.05786/) (2022)
- [JAKET: Joint Pre-training of Knowledge Graph and Language Understanding](https://mlsanity.com/paper/2010.00796/) (2020)
- [Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering](https://mlsanity.com/paper/2309.11206/) (2023)
- [Temporal Knowledge Graph Completion: A Survey](https://mlsanity.com/paper/2201.08236/) (2022)
- [TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph](https://mlsanity.com/paper/2104.07302/) (2021)
- [FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge](https://mlsanity.com/paper/2305.08281/) (2023)
- [RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding](https://mlsanity.com/paper/2212.01588/) (2022)
- [Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader](https://mlsanity.com/paper/1905.07098/) (2019)
- [Knowledge Base Question Answering by Case-based Reasoning over Subgraphs](https://mlsanity.com/paper/2202.10610/) (2022)
- [SKILL: Structured Knowledge Infusion for Large Language Models](https://mlsanity.com/paper/2205.08184/) (2022)
- [Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Task](https://mlsanity.com/paper/2310.06504/) (2023)

## Built on by (most cited first, 22 of 22)

- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Graph Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2408.08921/) (2024)
- [Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.20724/) (2024)
- [Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely](https://mlsanity.com/paper/2409.14924/) (2024)
- [A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges](https://mlsanity.com/paper/2412.11936/) (2024)
- [Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning](https://mlsanity.com/paper/2410.14211/) (2024)
- [AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering](https://mlsanity.com/paper/2510.05445/) (2025)
- [Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models](https://mlsanity.com/paper/2409.03155/) (2024)
- [Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey](https://mlsanity.com/paper/2504.10499/) (2025)
- [EffiQA: Efficient Question-Answering with Strategic Multi-Model Collaboration on Knowledge Graphs](https://mlsanity.com/paper/2406.01238/) (2024)
- [Large Language Models Can Better Understand Knowledge Graphs Than We Thought](https://mlsanity.com/paper/2402.11541/) (2024)
- [NG-Router: Graph-Supervised Multi-Agent Collaboration for Nutrition Question Answering](https://mlsanity.com/paper/2510.09854/) (2025)
- [ODA: Observation-Driven Agent for integrating LLMs and Knowledge Graphs](https://mlsanity.com/paper/2404.07677/) (2024)
- [Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need?](https://mlsanity.com/paper/2510.16582/) (2025)
- [LAD-RAG: Layout-aware Dynamic RAG for Visually-Rich Document Understanding](https://mlsanity.com/paper/2510.07233/) (2025)
- [Aligning Knowledge Graphs and Language Models for Factual Accuracy](https://mlsanity.com/paper/2507.13411/) (2025)
- [From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap](https://mlsanity.com/paper/2410.20791/) (2024)
- [Know3-RAG: A Knowledge-aware RAG Framework with Adaptive Retrieval, Generation, and Filtering](https://mlsanity.com/paper/2505.12662/) (2025)
- [A New Perspective on ADHD Research: Knowledge Graph Construction with LLMs and Network Based Insights](https://mlsanity.com/paper/2409.12853/) (2024)
- [The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)](https://mlsanity.com/paper/2505.10640/) (2025)
- [HyperComplEx: Adaptive Multi-Space Knowledge Graph Embeddings](https://mlsanity.com/paper/2511.10842/) (2025)
- [Improving LLM's Attachment to External Knowledge In Dialogue Generation Tasks Through Entity Anonymization](https://mlsanity.com/paper/2511.11946/) (2025)

## Abstract (arXiv)

Large language model (LLM) has achieved outstanding performance on various downstream tasks with its powerful natural language understanding and zero-shot capability, but LLM still suffers from knowledge limitation. Especially in scenarios that require long logical chains or complex reasoning, the hallucination and knowledge limitation of LLM limit its performance in question answering (QA). In this paper, we propose a novel framework KnowledgeNavigator to address these challenges by efficiently and accurately retrieving external knowledge from knowledge graph and using it as a key factor to enhance LLM reasoning. Specifically, KnowledgeNavigator first mines and enhances the potential constraints of the given question to guide the reasoning. Then it retrieves and filters external knowledge that supports answering through iterative reasoning on knowledge graph with the guidance of LLM and the question. Finally, KnowledgeNavigator constructs the structured knowledge into effective prompts that are friendly to LLM to help its reasoning. We evaluate KnowledgeNavigator on multiple public KGQA benchmarks, the experiments show the framework has great effectiveness and generalization, outperforming previous knowledge graph enhanced LLM methods and is comparable to the fully supervised models.