mlsanityData to 7 Oct 2026

KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs

arXiv 2504.07087 ↗First posted 9 Apr 2025cs.CL cs.AI cs.IRcited by 6

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Knowledge graphs have emerged as a popular method for injecting up-to-date, factual knowledge into large language models (LLMs). This is typically achieved by converting the knowledge graph into text that the LLM can process in context. While multiple methods of encoding knowledge graphs have been proposed, the impact of this textualization process on LLM performance remains under-explored. We introduce KG-LLM-Bench, a comprehensive and extensible benchmark spanning five knowledge graph understanding tasks, and evaluate how different encoding strategies affect performance across various base models. Our extensive experiments with seven language models and five textualization strategies provide insights for optimizing LLM performance on KG reasoning tasks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 52 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

18 of 52 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Causal Reasoning in Large Language Models: A Knowledge Graph Approach15 Oct
  2. 2024Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answering10 Oct
  3. 2024GraCoRe: Benchmarking Graph Comprehension and Complex Reasoning in Large Language Models3 Jul
  4. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities22 May
  5. 2023Can Language Models Solve Graph Problems in Natural Language?17 May

Other references · 13

  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  2. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  3. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 175
  4. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 84
  5. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 48
  6. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  7. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  8. 2024How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehensioncited by 30
  9. 2024Combining Knowledge Graphs and Large Language Modelscited by 16
  10. 2024Beyond Graphs: Can Large Language Models Comprehend Hypergraphs?cited by 11
  11. 2024CLR-Fact: Evaluating the Complex Logical Reasoning Capability of Large Language Models over Factual Knowledgecited by 9
  12. 2019WikiDataSets: Standardized sub-graphs from Wikidatacited by 5
1 more
  1. 2024G2T-LLM: Graph-to-Tree Text Encoding for Molecule Generation with Fine-Tuned Large Language Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzlescited by 1
  2. 2025Knowledge-Grounded Agentic Large Language Models for Multi-Hazard Understanding from Reconnaissance Reportscited by 1
  3. 2026Knowledge Graph Representations for LLM-Based Policy Compliance Reasoningcited by 1
  4. 2026MHGraphBench: Knowledge Graph-Grounded Benchmarking of Mental Health Knowledge in Large Language Modelscited by 0
  5. 2026Causal-Audit: Explicit and Auditable Graph-based Reasoning via Target-Aware Causal Chain Constructioncited by 0
  6. 2026GTA: Graph Theory Agent and Benchmark for Algorithmic Graph Reasoning with LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data
  2. 2025Grounding LLM Reasoning with Knowledge Graphs
  3. 2023KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models
  4. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmap
  5. 2023On Exploring the Reasoning Capability of Large Language Models with Knowledge Graphs
  6. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning
  7. 2025Injecting Knowledge Graphs into Large Language Models
  8. 2025Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs