mlsanityData to 7 Oct 2026

An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration

arXiv 2402.04978 ↗First posted 7 Feb 2024cs.CL cs.AIcited by 16

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

While Large Language Models (LLMs) demonstrate exceptional performance in a multitude of Natural Language Processing (NLP) tasks, they encounter challenges in practical applications, including issues with hallucinations, inadequate knowledge updating, and limited transparency in the reasoning process. To overcome these limitations, this study innovatively proposes a collaborative training-free reasoning scheme involving tight cooperation between Knowledge Graph (KG) and LLMs. This scheme first involves using LLMs to iteratively explore KG, selectively retrieving a task-relevant knowledge subgraph to support reasoning. The LLMs are then guided to further combine inherent implicit knowledge to reason on the subgraph while explicitly elucidating the reasoning process. Through such a cooperative approach, our scheme achieves more reliable knowledge-based reasoning and facilitates the tracing of the reasoning results. Experimental results show that our scheme significantly progressed across multiple datasets, notably achieving over a 10% improvement on the QALD10 dataset compared to the best baseline and the fine-tuned state-of-the-art (SOTA) work. Building on this success, this study hopes to offer a valuable reference for future research in the fusion of KG and LLMs, thereby enhancing LLMs' proficiency in solving complex issues.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 46 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

33 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Retrieval Augmentation for Commonsense Reasoning: A Unified Approach23 Oct

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  8. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  9. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  10. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  11. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  12. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
20 more
  1. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  2. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  3. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  4. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  5. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  6. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
  7. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  8. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  9. 2019PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Textcited by 182
  10. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 121
  11. 2021Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signalscited by 100
  12. 2021UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmarkcited by 98
  13. 2021CREAK: A Dataset for Commonsense Reasoning over Entity Knowledgecited by 88
  14. 2022DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Basescited by 83
  15. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Promptingcited by 77
  16. 2022Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answeringcited by 71
  17. 2022Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformercited by 45
  18. 2023Vera: A General-Purpose Plausibility Estimation Model for Commonsense Statementscited by 44
  19. 2022QALD-9-plus: A Multilingual Dataset for Question Answering over DBpedia and Wikidata Translated by Native Speakerscited by 39
  20. 2023ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examinationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 5; 2026: 3.

16 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  2. 2024AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agentscited by 100
  3. 2024Large Language Model Meets Graph Neural Network in Knowledge Distillationcited by 13
  4. 2024SMARTCAL: An Approach to Self-Aware Tool-Use Evaluation and Calibrationcited by 13
  5. 2025Biomedical Knowledge Graph: A Survey of Domains, Tasks, and Real-World Applicationscited by 6
  6. 2025Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentencescited by 5
  7. 2024Harnessing the Power of Semi-Structured Knowledge and LLMs with Triplet-Based Prefiltering for Question Answeringcited by 4
  8. 2024Combining LLMs and Knowledge Graphs to Reduce Hallucinations in Question Answeringcited by 3
  9. 2024Pyramid-Driven Alignment: Pyramid Principle Guided Integration of Large Language Models and Knowledge Graphscited by 2
  10. 2025Enhancing Large Language Models (LLMs) for Telecommunications using Knowledge Graphs and Retrieval-Augmented Generationcited by 1
  11. 2025Bonsai: Interpretable Tree-Adaptive Grounded Reasoningcited by 1
  12. 2025From Graph Retrieval to Schema Realization: Counterfactual Validation for Text-to-SPARQL over Heterogeneous Knowledge Graphscited by 1
4 more
  1. 2024GRL-Prompt: Towards Knowledge Graph based Prompt Optimization via Reinforcement Learningcited by 0
  2. 2026SAGE: Structure Aware Graph Expansion for Retrieval of Heterogeneous Datacited by 0
  3. 2026SchemaRAG: Dynamic Large Schema Reduction for LLM-driven Structured Information Extractioncited by 0
  4. 2026AGE: Adaptive-masking for Graph Embedding in Graph Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data
  2. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities
  3. 2025Grounding LLM Reasoning with Knowledge Graphs
  4. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph
  5. 2025Learn to Think: Bootstrapping LLM Reasoning Capability Through Graph Representation Learning
  6. 2024LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs
  7. 2025Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoning
  8. 2025Reliable Reasoning Path: Distilling Effective Guidance for LLM Reasoning with Knowledge Graphs