mlsanityData to 7 Oct 2026

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

arXiv 2307.07697 ↗First posted 15 Jul 2023cs.CLcited by 235

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Although large language models (LLMs) have achieved significant success in various tasks, they often struggle with hallucination problems, especially in scenarios requiring deep and responsible reasoning. These issues could be partially addressed by introducing external knowledge graphs (KG) in LLM reasoning. In this paper, we propose a new LLM-KG integrating paradigm ``hboxLLMotimeshboxKG'' which treats the LLM as an agent to interactively explore related entities and relations on KGs and perform reasoning based on the retrieved knowledge. We further implement this paradigm by introducing a new approach called Think-on-Graph (ToG), in which the LLM agent iteratively executes beam search on KG, discovers the most promising reasoning paths, and returns the most likely reasoning results. We use a number of well-designed experiments to examine and illustrate the following advantages of ToG: 1) compared with LLMs, ToG has better deep reasoning power; 2) ToG has the ability of knowledge traceability and knowledge correctability by leveraging LLMs reasoning and expert feedback; 3) ToG provides a flexible plug-and-play framework for different LLMs, KGs and prompting strategies without any additional training cost; 4) the performance of ToG with small LLM models could exceed large LLM such as GPT-4 in certain scenarios and this reduces the cost of LLM deployment and application. As a training-free method with lower computational cost and better generality, ToG achieves overall SOTA in 6 out of 9 datasets where most previous SOTAs rely on additional training.

Key facts. Cited by 235 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 52 of its 87 references that are in the map. First posted on arXiv 15 Jul 2023.

Builds on

52 of 87 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Retrieval Augmentation for Commonsense Reasoning: A Unified Approach23 Oct

Other references · 51

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
38 more
  1. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  2. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  3. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  4. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  5. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  6. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  7. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  8. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  9. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  10. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  11. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  12. 2019Knowledge Enhanced Contextual Word Representationscited by 294
  13. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  14. 2019PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Textcited by 182
  15. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 169
  16. 2015Large-scale Simple Question Answering with Memory Networkscited by 127
  17. 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
  18. 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
  19. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  20. 2021Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signalscited by 100
  21. 2021CREAK: A Dataset for Commonsense Reasoning over Entity Knowledgecited by 88
  22. 2022DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Basescited by 83
  23. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Promptingcited by 77
  24. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
  25. 2023Few-shot In-context Learning for Knowledge Base Question Answeringcited by 74
  26. 2021Poolingformer: Long Document Modeling with Pooling Attentioncited by 60
  27. 2022TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Basescited by 59
  28. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  29. 2021Complex Knowledge Base Question Answering: A Surveycited by 52
  30. 2022QALD-9-plus: A Multilingual Dataset for Question Answering over DBpedia and Wikidata Translated by Native Speakerscited by 39
  31. 2021Program Transfer for Answering Complex Questions over Knowledge Basescited by 34
  32. 2022FiDO: Fusion-in-Decoder optimized for stronger performance and faster inferencecited by 31
  33. 2022SKILL: Structured Knowledge Infusion for Large Language Modelscited by 27
  34. 2023Direct Fact Retrieval from Knowledge Graphs without Entity Linkingcited by 22
  35. 2022Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Databasecited by 16
  36. 2023Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Modelscited by 16
  37. 2022Knowledge Prompts: Injecting World Knowledge into Language Models through Soft Promptscited by 6
  38. 2023GETT-QA: Graph Embedding based T2T Transformer for Knowledge Graph Question Answeringcited by 6

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 36; 2025: 101; 2026: 92.

235 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  3. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  4. 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoningcited by 90
  5. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  6. 2024Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generationcited by 63
  7. 2025PathRAG: Pruning Graph-based Retrieval Augmented Generation with Relational Pathscited by 60
  8. 2024Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answeringcited by 59
  9. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 50
  10. 2024Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMscited by 48
  11. 2024MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPscited by 45
  12. 2025In-depth Analysis of Graph-based RAG in a Unified Frameworkcited by 45
38 more
  1. 2023ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Modelscited by 44
  2. 2024Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoningcited by 44
  3. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 40
  4. 2024FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answeringcited by 38
  5. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  6. 2024SPINACH: SPARQL-Based Information Navigation for Challenging Real-World Questionscited by 27
  7. 2024HybGRAG: Hybrid Retrieval-Augmented Generation on Textual and Relational Knowledge Basescited by 26
  8. 2023HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responsescited by 25
  9. 2024Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Modelscited by 25
  10. 2024Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrievalcited by 24
  11. 2024Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentationcited by 24
  12. 2025AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corporacited by 23
  13. 2024Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query Perspectivecited by 20
  14. 2023Graph Meets LLMs: Towards Large Graph Modelscited by 19
  15. 2024GraphOTTER: Evolving LLM-based Graph Reasoning for Complex Table Question Answeringcited by 18
  16. 2025Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Modelscited by 16
  17. 2025Graph RAG-Tool Fusioncited by 15
  18. 2025Tool-to-Agent Retrieval: Bridging Tools and Agents for Scalable LLM Multi-Agent Systemscited by 15
  19. 2023Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safetycited by 14
  20. 2025Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoningcited by 14
  21. 2024EWEK-QA: Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systemscited by 13
  22. 2025VistaWise: Building Cost-Effective Agent with Cross-Modal Knowledge Graph for Minecraftcited by 13
  23. 2024GraphEval36K: Benchmarking Coding and Reasoning Capabilities of Large Language Models on Graph Datasetscited by 12
  24. 2025GraphCheck: Breaking Long-Term Text Barriers with Extracted Knowledge Graph-Powered Fact-Checkingcited by 12
  25. 2025KiRAG: Knowledge-Driven Iterative Retriever for Enhancing Retrieval-Augmented Generationcited by 12
  26. 2025Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Modelscited by 11
  27. 2023Graph Elicitation for Guiding Multi-Step Reasoning in Large Language Modelscited by 10
  28. 2024Graph Neural Network Enhanced Retrieval for Question Answering of LLMscited by 10
  29. 2025Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrievalcited by 10
  30. 2025Scaling Graph Chain-of-Thought Reasoning: A Multi-Agent Framework with Efficient LLM Servingcited by 10
  31. 2024Temporal Knowledge Graph Question Answering: A Surveycited by 9
  32. 20241+1>2: Can Large Language Models Serve as Cross-Lingual Knowledge Aggregators?cited by 9
  33. 2025Mixture of Structural-and-Textual Retrieval over Text-rich Graph Knowledge Basescited by 9
  34. 2025SAKE: Structured Agentic Knowledge Extrapolation for Complex LLM Reasoning via Reinforcement Learningcited by 9
  35. 2025Reliable Reasoning Path: Distilling Effective Guidance for LLM Reasoning with Knowledge Graphscited by 9
  36. 2025What Breaks Knowledge Graph based RAG? Benchmarking and Empirical Insights into Reasoning under Incomplete Knowledgecited by 9
  37. 2025NG-Router: Graph-Supervised Multi-Agent Collaboration for Nutrition Question Answeringcited by 9
  38. 2024Enhancing Large Language Models with Pseudo- and Multisource- Knowledge Graphs for Open-ended Question Answeringcited by 8

50 most cited of 235 All 235 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning
  2. 2025Grounding LLM Reasoning with Knowledge Graphs
  3. 2023On Exploring the Reasoning Capability of Large Language Models with Knowledge Graphs
  4. 2024Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models
  5. 2025KnowPath: Knowledge-enhanced Reasoning via LLM-generated Inference Paths over Knowledge Graphs
  6. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data
  7. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
  8. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities