mlsanityData to 14 Oct 2026

Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning

arXiv 2505.24478 ↗First posted 30 May 2025cs.AI cs.CLcited by 17

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Integrating Large Language Models (LLMs) with Knowledge Graphs (KGs) results in complex systems with numerous hyperparameters that directly affect performance. While such systems are increasingly common in retrieval-augmented generation, the role of systematic hyperparameter optimization remains underexplored. In this paper, we study this problem in the context of Cognee, a modular framework for end-to-end KG construction and retrieval. Using three multi-hop QA benchmarks (HotPotQA, TwoWikiMultiHop, and MuSiQue) we optimize parameters related to chunking, graph construction, retrieval, and prompting. Each configuration is scored using established metrics (exact match, F1, and DeepEval's LLM-based correctness metric). Our results demonstrate that meaningful gains can be achieved through targeted tuning. While the gains are consistent, they are not uniform, with performance varying across datasets and metrics. This variability highlights both the value of tuning and the limitations of standard evaluation measures. While demonstrating the immediate potential of hyperparameter tuning, we argue that future progress will depend not only on architectural advances but also on clearer frameworks for optimization and evaluation in complex, modular systems.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 42 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

39 of 42 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec
  2. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning2 Oct
  3. 2023Complex Logical Reasoning over Knowledge Graphs using Large Language Models2 May

Other references · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  5. 2019Language Models as Knowledge Bases?cited by 1,936
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  7. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  8. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  9. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  11. 2023BloombergGPT: A Large Language Model for Financecited by 939
  12. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
24 more
  1. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  2. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  3. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  4. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 410
  5. 2018From Louvain to Leiden: guaranteeing well-connected communitiescited by 385
  6. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  7. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  8. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 281
  9. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 223
  10. 2019Neural Legal Judgment Prediction in Englishcited by 209
  11. 2024Corrective Retrieval Augmented Generationcited by 197
  12. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 172
  13. 2017Variational Reasoning for Question Answering with Knowledge Graphcited by 170
  14. 2023Knowledge Graph Prompting for Multi-Document Question Answeringcited by 152
  15. 2023The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4cited by 112
  16. 2022Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answeringcited by 93
  17. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 88
  18. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directionscited by 70
  19. 2023Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inferencecited by 40
  20. 2025Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learningcited by 39
  21. 2024Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspectivecited by 29
  22. 2024AutoRAG-HP: Automatic Online Hyper-Parameter Tuning for Retrieval-Augmented Generationcited by 13
  23. 2025Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systemscited by 12
  24. 2024Introducing a new hyper-parameter for RAG: Context Window Utilizationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 12.

17 papers.

Sort:

By area:

  1. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 254
  2. 2025MIRIX: Multi-Agent Memory System for LLM-Based Agentscited by 173
  3. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  4. 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 90
  5. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 25
  6. 2026GraphAgents: Knowledge Graph-Guided Agentic AI for Cross-Domain Materials Designcited by 9
  7. 2025Everything is Context: Agentic File System Abstraction for Context Engineeringcited by 6
  8. 2025Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledgecited by 3
  9. 2026Think Locally, Explain Globally: Graph-Guided LLM Investigations via Local Reasoning and Belief Propagationcited by 3
  10. 2026Structured Belief State and the First Precision-Aware Benchmark for LLM Memory Retrievalcited by 3
  11. 2026Cost and Accuracy of Long-Term Memory in Distributed Multi-Agent Systems Based on Large Language Modelscited by 2
  12. 2026ZenBrain: A Neuroscience-Inspired 7-Layer Memory Architecture for Autonomous AI Systemscited by 1
5 more
  1. 2026EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memorycited by 1
  2. 2026Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Datacited by 0
  3. 2026Know It, Act on It: Investigating Memory Utilization in LLM Personalizationcited by 0
  4. 2026Towards Fully Automated Medical Imaging Code Generation via Validation-based Context Engineeringcited by 0
  5. 2026Probing Stability-Plasticity Tradeoffs in Agent Memory through Cognitive Experimental Paradigmscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation
  2. 2026A Human-in-the-Loop, LLM-Centered Architecture for Knowledge-Graph Question Answering
  3. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data
  4. 2025Inference Scaled GraphRAG: Improving Multi Hop Question Answering on Knowledge Graphs
  5. 2025KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs
  6. 2026GraphScout: Empowering Large Language Models with Intrinsic Exploration Ability for Agentic Graph Reasoning
  7. 2025KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration
  8. 2025GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation