# Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs

arXiv: https://arxiv.org/abs/2406.14282 · HTML: https://mlsanity.com/paper/2406.14282/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 33 of its 53 references that are in the map.
- First posted on arXiv 20 Jun 2024.

## Placed in

- Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/)

## Builds on (most cited first, 30 of 33)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://mlsanity.com/paper/2305.15294/) (2023)
- [Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://mlsanity.com/paper/2310.01061/) (2023)
- [FireAct: Toward Language Agent Fine-tuning](https://mlsanity.com/paper/2310.05915/) (2023)
- [RA-DIT: Retrieval-Augmented Dual Instruction Tuning](https://mlsanity.com/paper/2310.01352/) (2023)
- [Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings](https://mlsanity.com/paper/2002.05969/) (2020)
- [RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation](https://mlsanity.com/paper/2404.00610/) (2024)
- [Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs](https://mlsanity.com/paper/2010.11465/) (2020)
- [Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources](https://mlsanity.com/paper/2305.13269/) (2023)
- [Embedding Logical Queries on Knowledge Graphs](https://mlsanity.com/paper/1806.01445/) (2018)
- [Large Language Models Can Learn Temporal Reasoning](https://mlsanity.com/paper/2401.06853/) (2024)
- [Complex Query Answering with Neural Link Predictors](https://mlsanity.com/paper/2011.03459/) (2020)
- [ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent](https://mlsanity.com/paper/2312.10003/) (2023)

## Built on by (most cited first, 24 of 24)

- [ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning](https://mlsanity.com/paper/2503.19470/) (2025)
- [KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation](https://mlsanity.com/paper/2409.13731/) (2024)
- [Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning](https://mlsanity.com/paper/2410.12952/) (2024)
- [Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs](https://mlsanity.com/paper/2507.09477/) (2025)
- [Large Language Models for Planning: A Comprehensive and Systematic Survey](https://mlsanity.com/paper/2505.19683/) (2025)
- [Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities](https://mlsanity.com/paper/2505.20099/) (2025)
- [Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrieval](https://mlsanity.com/paper/2410.13765/) (2024)
- [Test-Time Adaptation for LLM Agents via Environment Interaction](https://mlsanity.com/paper/2511.04847/) (2025)
- [Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering](https://mlsanity.com/paper/2502.11491/) (2025)
- [RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answering](https://mlsanity.com/paper/2410.22353/) (2024)
- [Credible Plan-Driven RAG Method for Multi-Hop Question Answering](https://mlsanity.com/paper/2504.16787/) (2025)
- [Large Language Models for Knowledge Graph Embedding: A Survey](https://mlsanity.com/paper/2501.07766/) (2025)
- [From Symbolic to Neural and Back: Exploring Knowledge Graph-Large Language Model Synergies](https://mlsanity.com/paper/2506.09566/) (2025)
- [Only Ask What You Don't Know: Grounded Delta Planning for Efficient Multi-step RAG](https://mlsanity.com/paper/2606.22681/) (2026)
- [GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion](https://mlsanity.com/paper/2502.11471/) (2025)
- [Enhancing LLM Tool Use with High-quality Instruction Data from Knowledge Graph](https://mlsanity.com/paper/2506.21071/) (2025)
- [A Survey of Query Optimization in Large Language Models](https://mlsanity.com/paper/2412.17558/) (2024)
- [KunLunBaizeRAG: Reinforcement Learning Driven Inference Performance Leap for Large Language Models](https://mlsanity.com/paper/2506.19466/) (2025)
- [Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided Decomposition](https://mlsanity.com/paper/2509.07555/) (2025)
- [VulReaD: Knowledge-Graph-guided Software Vulnerability Reasoning and Detection](https://mlsanity.com/paper/2602.10787/) (2026)
- [WebClipper: Efficient Evolution of Web Agents with Graph-based Trajectory Pruning](https://mlsanity.com/paper/2602.12852/) (2026)
- [RealRoute: Dynamic Query Routing System via Retrieve-then-Verify Paradigm](https://mlsanity.com/paper/2604.20860/) (2026)
- [LLMs+Graphs: Toward Graph-Native, Synergistic AI Systems](https://mlsanity.com/paper/2606.11560/) (2026)
- [KARMA: Knowledge graph-based Automated Reasoning Materialization and Alignment](https://mlsanity.com/paper/2607.03166/) (2026)

## Abstract (arXiv)

Improving the performance of large language models (LLMs) in complex question-answering (QA) scenarios has always been a research focal point. Recent studies have attempted to enhance LLMs' performance by combining step-wise planning with external retrieval. While effective for advanced models like GPT-3.5, smaller LLMs face challenges in decomposing complex questions, necessitating supervised fine-tuning. Previous work has relied on manual annotation and knowledge distillation from teacher LLMs, which are time-consuming and not accurate enough. In this paper, we introduce a novel framework for enhancing LLMs' planning capabilities by using planning data derived from knowledge graphs (KGs). LLMs fine-tuned with this data have improved planning capabilities, better equipping them to handle complex QA tasks that involve retrieval. Evaluations on multiple datasets, including our newly proposed benchmark, highlight the effectiveness of our framework and the benefits of KG-derived planning data.