mlsanityData to 7 Oct 2026

Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data

arXiv 2412.10654 ↗First posted 14 Dec 2024cs.CL cs.LGcited by 6

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, they often struggle with complex reasoning tasks and are prone to hallucination. Recent research has shown promising results in leveraging knowledge graphs (KGs) to enhance LLM performance. KGs provide a structured representation of entities and their relationships, offering a rich source of information that can enhance the reasoning capabilities of LLMs. For this work, we have developed different techniques that tightly integrate KG structures and semantics into LLM representations. Our results show that we are able to significantly improve the performance of LLMs in complex reasoning scenarios, and ground the reasoning process with KGs. We are the first to represent KGs with programming language and fine-tune pretrained LLMs with KGs. This integration facilitates more accurate and interpretable reasoning processes, paving the way for more advanced reasoning capabilities of LLMs.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 40 references that are in the map. First posted on arXiv 14 Dec 2024.

Builds on

25 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?26 Feb
  2. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment13 Feb
  3. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Model9 Oct
  4. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning2 Oct
  5. 2023Can Language Models Solve Graph Problems in Natural Language?17 May

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  5. 2023Language is All a Graph Needscited by 219
  6. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  7. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  8. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  9. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
  10. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 94
  11. 2023At Which Training Stage Does Code Data Help LLMs Reasoning?cited by 82
  12. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
8 more
  1. 2024To Code, or Not To Code? Exploring Impact of Code in Pre-trainingcited by 47
  2. 2024Unveiling the Impact of Coding Data Instruction Fine-Tuning on Large Language Models Reasoningcited by 31
  3. 2024SPARQL Generation: an analysis on fine-tuning OpenLLaMA for Question Answering over a Life Science Knowledge Graphcited by 19
  4. 2023Code-Style In-Context Learning for Knowledge-Based Question Answeringcited by 13
  5. 2024Leveraging small language models for Text2SPARQL tasks to improve the resilience of AI assistancecited by 13
  6. 2024GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encodingcited by 11
  7. 2024SPARQL Generation with Entity Pre-trained GPT for KG Question Answeringcited by 6
  8. 2024Investigating Instruction Tuning Large Language Models on Graphscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  2. 2025On Verifiable Legal Reasoning: A Multi-Agent Framework with Formalized Knowledge Representationscited by 16
  3. 2025Aligning Knowledge Graphs and Language Models for Factual Accuracycited by 4
  4. 2026Knowledge Graph Representations for LLM-Based Policy Compliance Reasoningcited by 1
  5. 2026Structure First, Reason Next: Enhancing a Large Language Model using Knowledge Graph for Numerical Reasoning in Financial Documentscited by 0
  6. 2026Can Structural Cues Save LLMs? Evaluating Language Models in Massive Document Streamscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Grounding LLM Reasoning with Knowledge Graphs
  2. 2025KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs
  3. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities
  4. 2023On Exploring the Reasoning Capability of Large Language Models with Knowledge Graphs
  5. 2025LLM-empowered knowledge graph construction: A survey
  6. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmap
  7. 2025Reliable Reasoning Path: Distilling Effective Guidance for LLM Reasoning with Knowledge Graphs
  8. 2025From Symbolic to Neural and Back: Exploring Knowledge Graph-Large Language Model Synergies