# Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation

arXiv: https://arxiv.org/abs/2407.10805 · HTML: https://mlsanity.com/paper/2407.10805/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/)

## Builds on (most cited first, 24 of 24)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://mlsanity.com/paper/2305.15294/) (2023)
- [Benchmarking Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2309.01431/) (2023)
- [Knowledge Enhanced Contextual Word Representations](https://mlsanity.com/paper/1909.04164/) (2019)
- [Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph](https://mlsanity.com/paper/2307.07697/) (2023)
- [Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models](https://mlsanity.com/paper/2311.09210/) (2023)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources](https://mlsanity.com/paper/2305.13269/) (2023)
- [CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge](https://mlsanity.com/paper/2109.01653/) (2021)
- [Learning to Filter Context for Retrieval-Augmented Generation](https://mlsanity.com/paper/2311.08377/) (2023)
- [SPARQA: Skeleton-based Semantic Parsing for Complex Questions over Knowledge Bases](https://mlsanity.com/paper/2003.13956/) (2020)
- [Entropy-Based Decoding for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2406.17519/) (2024)
- [Efficient Knowledge Infusion via KG-LLM Alignment](https://mlsanity.com/paper/2406.03746/) (2024)
- [Joint Multi-Facts Reasoning Network For Complex Temporal Question Answering Over Knowledge Graph](https://mlsanity.com/paper/2401.02212/) (2024)
- [The Integration of Semantic and Structural Knowledge in Knowledge Graph Entity Typing](https://mlsanity.com/paper/2404.08313/) (2024)

## Built on by (most cited first, 30 of 64)

- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [Synergizing RAG and Reasoning: A Systematic Review](https://mlsanity.com/paper/2504.15909/) (2025)
- [Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities](https://mlsanity.com/paper/2506.18019/) (2025)
- [Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agents](https://mlsanity.com/paper/2601.22311/) (2026)
- [A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworks](https://mlsanity.com/paper/2501.10069/) (2025)
- [Reasoning of Large Language Models over Knowledge Graphs with Super-Relations](https://mlsanity.com/paper/2503.22166/) (2025)
- [RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generation](https://mlsanity.com/paper/2502.10996/) (2025)
- [LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data](https://mlsanity.com/paper/2502.12583/) (2025)
- [MemoTime: Memory-Augmented Temporal Knowledge Graph Enhanced Large Language Model Reasoning](https://mlsanity.com/paper/2510.13614/) (2025)
- [PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments](https://mlsanity.com/paper/2603.23231/) (2026)
- [Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrieval](https://mlsanity.com/paper/2509.21710/) (2025)
- [Multi-agent systems for chemical engineering: A review and perspective](https://mlsanity.com/paper/2508.07880/) (2025)
- [Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding](https://mlsanity.com/paper/2510.15253/) (2025)
- [InfMem: Learning System-2 Memory Control for Long-Context Agent](https://mlsanity.com/paper/2602.02704/) (2026)
- [Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need?](https://mlsanity.com/paper/2510.16582/) (2025)
- [GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databases](https://mlsanity.com/paper/2504.05478/) (2025)
- [AutoGraph-R1: End-to-End Reinforcement Learning for Knowledge Graph Construction](https://mlsanity.com/paper/2510.15339/) (2025)
- [TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework](https://mlsanity.com/paper/2511.05385/) (2025)
- [DAMR: Efficient and Adaptive Context-Aware Knowledge Graph Question Answering with LLM-Guided MCTS](https://mlsanity.com/paper/2508.00719/) (2025)
- [GraphWalker: Agentic Knowledge Graph Question Answering via Synthetic Trajectory Curriculum](https://mlsanity.com/paper/2603.28533/) (2026)
- [DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation](https://mlsanity.com/paper/2506.01954/) (2025)
- [KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering](https://mlsanity.com/paper/2509.04716/) (2025)
- [Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models](https://mlsanity.com/paper/2505.00979/) (2025)
- [KG-TRACES: Enhancing Large Language Models with Knowledge Graph-constrained Trajectory Reasoning and Attribution Supervision](https://mlsanity.com/paper/2506.00783/) (2025)
- [Enhancing Large Language Models with Reward-guided Tree Search for Knowledge Graph Question and Answering](https://mlsanity.com/paper/2505.12476/) (2025)
- [MultiRAG: A Knowledge-guided Framework for Mitigating Hallucination in Multi-source Retrieval Augmented Generation](https://mlsanity.com/paper/2508.03553/) (2025)
- [CLAUSE: Agentic Neuro-Symbolic Knowledge Graph Reasoning via Dynamic Learnable Context Engineering](https://mlsanity.com/paper/2509.21035/) (2025)
- [GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2509.22009/) (2025)
- [KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering](https://mlsanity.com/paper/2512.10999/) (2025)
- [Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking](https://mlsanity.com/paper/2603.00267/) (2026)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) has improved large language models (LLMs) by using knowledge retrieval to overcome knowledge deficiencies. However, current RAG methods often fall short of ensuring the depth and completeness of retrieved information, which is necessary for complex reasoning tasks. In this work, we introduce Think-on-Graph 2.0 (ToG-2), a hybrid RAG framework that iteratively retrieves information from both unstructured and structured knowledge sources in a tight-coupling manner. Specifically, ToG-2 leverages knowledge graphs (KGs) to link documents via entities, facilitating deep and knowledge-guided context retrieval. Simultaneously, it utilizes documents as entity contexts to achieve precise and efficient graph retrieval. ToG-2 alternates between graph retrieval and context retrieval to search for in-depth clues relevant to the question, enabling LLMs to generate answers. We conduct a series of well-designed experiments to highlight the following advantages of ToG-2: 1) ToG-2 tightly couples the processes of context retrieval and graph retrieval, deepening context retrieval via the KG while enabling reliable graph retrieval based on contexts; 2) it achieves deep and faithful reasoning in LLMs through an iterative knowledge retrieval process of collaboration between contexts and the KG; and 3) ToG-2 is training-free and plug-and-play compatible with various LLMs. Extensive experiments demonstrate that ToG-2 achieves overall state-of-the-art (SOTA) performance on 6 out of 7 knowledge-intensive datasets with GPT-3.5, and can elevate the performance of smaller models (e.g., LLAMA-2-13B) to the level of GPT-3.5's direct reasoning. The source code is available on https://github.com/IDEA-FinAI/ToG-2.