mlsanityData to 14 Oct 2026

On Exploring the Reasoning Capability of Large Language Models with Knowledge Graphs

arXiv 2312.00353 ↗First posted 1 Dec 2023cs.CL cs.AIcited by 1

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

This paper examines the capacity of LLMs to reason with knowledge graphs using their internal knowledge graph, i.e., the knowledge graph they learned during pre-training. Two research questions are formulated to investigate the accuracy of LLMs in recalling information from pre-training knowledge graphs and their ability to infer knowledge graph relations from context. To address these questions, we employ LLMs to perform four distinct knowledge graph reasoning tasks. Furthermore, we identify two types of hallucinations that may occur during knowledge reasoning with LLMs: content and ontology hallucination. Our experimental results demonstrate that LLMs can successfully tackle both simple and complex knowledge graph reasoning tasks from their own memory, as well as infer from input context.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 22 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

17 of 22 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions20 Dec
  2. 2022Large Language Models are few(1)-shot Table Reasoners13 Oct

Other references · 15

  1. 2022Emergent Abilities of Large Language Modelscited by 2,512
  2. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  3. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  4. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  5. 2022A Survey on In-context Learningcited by 721
  6. 2022Large Language Models Can Self-Improvecited by 664
  7. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  8. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  9. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  10. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 334
  11. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  12. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
3 more
  1. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  2. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 112
  3. 2022Inferring Implicit Relations in Complex Questions with Language Modelscited by 9

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Protoknowledge Shapes Behaviour of LLMs in Downstream Tasks: Memorization and Generalization with Knowledge Graphscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data
  2. 2025Grounding LLM Reasoning with Knowledge Graphs
  3. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph
  4. 2025KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs
  5. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning
  6. 2025Reliable Reasoning Path: Distilling Effective Guidance for LLM Reasoning with Knowledge Graphs
  7. 2024Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models
  8. 2025KnowPath: Knowledge-enhanced Reasoning via LLM-generated Inference Paths over Knowledge Graphs