# Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering

arXiv: https://arxiv.org/abs/2601.11255 · HTML: https://mlsanity.com/paper/2601.11255/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 28 of its 53 references that are in the map.
- First posted on arXiv 16 Jan 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/)

## Builds on (most cited first, 28 of 28)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024)
- [DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models](https://mlsanity.com/paper/2309.03883/) (2023)
- [Corrective Retrieval Augmented Generation](https://mlsanity.com/paper/2401.15884/) (2024)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG](https://mlsanity.com/paper/2410.05983/) (2024)
- [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020)
- [Inference Scaling for Long-Context Retrieval Augmented Generation](https://mlsanity.com/paper/2410.04343/) (2024)
- [Why Does the Effective Context Length of LLMs Fall Short?](https://mlsanity.com/paper/2410.18745/) (2024)
- [LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering](https://mlsanity.com/paper/2410.18050/) (2024)
- [Chain-of-Retrieval Augmented Generation](https://mlsanity.com/paper/2501.14342/) (2025)
- [From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging](https://mlsanity.com/paper/2410.01215/) (2024)
- [LongCodeZip: Compress Long Context for Code Language Models](https://mlsanity.com/paper/2510.00446/) (2025)
- [DeepRAG: Thinking to Retrieve Step by Step for Large Language Models](https://mlsanity.com/paper/2502.01142/) (2025)
- [SWE-QA: Can Language Models Answer Repository-level Code Questions?](https://mlsanity.com/paper/2509.14635/) (2025)
- [JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking](https://mlsanity.com/paper/2411.00142/) (2024)
- [Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisal](https://mlsanity.com/paper/2508.05988/) (2025)
- [SiReRAG: Indexing Similar and Related Information for Multihop Reasoning](https://mlsanity.com/paper/2412.06206/) (2024)
- [Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering](https://mlsanity.com/paper/2502.14245/) (2025)
- [MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search](https://mlsanity.com/paper/2503.20757/) (2025)
- [AttentionRAG: Attention-Guided Context Pruning in Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10720/) (2025)
- [EvoC2Rust: A Skeleton-guided Framework for Project-Level C-to-Rust Translation](https://mlsanity.com/paper/2508.04295/) (2025)
- [LastingBench: Defend Benchmarks Against Knowledge Leakage](https://mlsanity.com/paper/2506.21614/) (2025)
- [Plan*RAG: Efficient Test-Time Planning for Retrieval Augmented Generation](https://mlsanity.com/paper/2410.20753/) (2024)
- [FlowMalTrans: Unsupervised Binary Code Translation for Malware Detection Using Flow-Adapter Architecture](https://mlsanity.com/paper/2508.20212/) (2025)
- [Robust Preference Alignment via Directional Neighborhood Consensus](https://mlsanity.com/paper/2510.20498/) (2025)
- [Beyond Chat: a Framework for LLMs as Human-Centered Support Systems](https://mlsanity.com/paper/2511.03729/) (2025)

## Built on by (most cited first, 14 of 14)

- [SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents](https://mlsanity.com/paper/2601.16746/) (2026)
- [Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis](https://mlsanity.com/paper/2603.16877/) (2026)
- [Seeing is Coding: On the Effectiveness of Vision Language Models in Code Understanding](https://mlsanity.com/paper/2602.01785/) (2026)
- [Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-off](https://mlsanity.com/paper/2603.26815/) (2026)
- [Improving the Completeness and Comparability of Segment Disclosures: A Large Language Model Approach](https://mlsanity.com/paper/2605.23924/) (2026)
- [SWE-Pruner Pro: The Coder LLM Already Knows What to Prune](https://mlsanity.com/paper/2607.18213/) (2026)
- [Beyond the Reranker: Do RAG Retrieval Enhancements Help Once a Strong Reranker Is Present?](https://mlsanity.com/paper/2606.28367/) (2026)
- [BaziQA-Benchmark: Evaluating Symbolic and Temporally Compositional Reasoning in Large Language Models](https://mlsanity.com/paper/2602.12889/) (2026)
- [LLM Reasoning Is Latent, Not the Chain of Thought](https://mlsanity.com/paper/2604.15726/) (2026)
- [Integrating Chain-of-Thought into Generative Retrieval: A Preliminary Study](https://mlsanity.com/paper/2605.22358/) (2026)
- [Can Hallucinations Be Useful? Solving Multi-Hop Questions With SLMs By Chaining System-I/II Reasoning](https://mlsanity.com/paper/2605.27596/) (2026)
- [ChartWalker: Benchmarking the Cross-Chart RAG Task with Hierarchical Knowledge Graphs](https://mlsanity.com/paper/2606.23997/) (2026)
- [HyGRL: Adaptive Hybrid Graph Reasoning for Multi-Entity Questions](https://mlsanity.com/paper/2607.19398/) (2026)
- [PANINI-DR: Controlled Deep Research through Generative Semantic Workspaces](https://mlsanity.com/paper/2610.15208/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has demonstrated significant effectiveness in enhancing large language models (LLMs) for complex multi-hop question answering (QA). For multi-hop QA tasks, current iterative approaches predominantly rely on LLMs to self-guide and plan multi-step exploration paths during retrieval, leading to substantial challenges in maintaining reasoning coherence across steps from inaccurate query decomposition and error propagation. To address these issues, we introduce Reasoning Tree Guided RAG (RT-RAG), a novel hierarchical framework for complex multi-hop QA. RT-RAG systematically decomposes multi-hop questions into explicit reasoning trees, minimizing inaccurate decomposition through structured entity analysis and consensus-based tree selection that clearly separates core queries, known entities, and unknown entities. Subsequently, a bottom-up traversal strategy employs iterative query rewriting and refinement to collect high-quality evidence, thereby mitigating error propagation. Comprehensive experiments show that RT-RAG substantially outperforms state-of-the-art methods by 7.0% F1 and 6.0% EM, demonstrating the effectiveness of RT-RAG in complex multi-hop QA.