# Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering

arXiv: https://arxiv.org/abs/2502.14245 · HTML: https://mlsanity.com/paper/2502.14245/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Placed in

- Retrieval-augmented generation / [Knowledge-graph and structured-knowledge RAG](https://mlsanity.com/rag/knowledge-graph-and-structured-knowledge-rag/)

## Builds on (most cited first, 20 of 20)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [Learning to Compress Prompts with Gist Tokens](https://mlsanity.com/paper/2304.08467/) (2023)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://mlsanity.com/paper/2305.15294/) (2023)
- [Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://mlsanity.com/paper/2310.01061/) (2023)
- [Recursively Summarizing Books with Human Feedback](https://mlsanity.com/paper/2109.10862/) (2021)
- [LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs](https://mlsanity.com/paper/2406.15319/) (2024)
- [Dr.ICL: Demonstration-Retrieved In-context Learning](https://mlsanity.com/paper/2305.14128/) (2023)
- [Complex QA and language models hybrid architectures, Survey](https://mlsanity.com/paper/2302.09051/) (2023)
- [OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs](https://mlsanity.com/paper/2409.05152/) (2024)
- [KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation](https://mlsanity.com/paper/2403.14950/) (2024)

## Built on by (most cited first, 22 of 22)

- [Synergizing RAG and Reasoning: A Systematic Review](https://mlsanity.com/paper/2504.15909/) (2025)
- [MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning](https://mlsanity.com/paper/2511.02805/) (2025)
- [Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering](https://mlsanity.com/paper/2601.11255/) (2026)
- [DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections](https://mlsanity.com/paper/2508.15851/) (2025)
- [When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning](https://mlsanity.com/paper/2505.15276/) (2025)
- [Curriculum Guided Reinforcement Learning for Efficient Multi Hop Retrieval Augmented Generation](https://mlsanity.com/paper/2505.17391/) (2025)
- [GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning](https://mlsanity.com/paper/2510.20548/) (2025)
- [Think Before You Retrieve: Learning Test-Time Adaptive Search with Small Language Models](https://mlsanity.com/paper/2511.07581/) (2025)
- [TreeHop: Efficient Embedding-Level Query Rewriter](https://mlsanity.com/paper/2504.20114/) (2025)
- [HetaRAG: Hybrid Deep Retrieval-Augmented Generation across Heterogeneous Data Stores](https://mlsanity.com/paper/2509.21336/) (2025)
- [StepChain GraphRAG: Reasoning Over Knowledge Graphs for Multi-Hop Question Answering](https://mlsanity.com/paper/2510.02827/) (2025)
- [How does Watermarking Affect Visual Language Models in Document Understanding?](https://mlsanity.com/paper/2504.01048/) (2025)
- [MuISQA: Multi-Intent Retrieval-Augmented Generation for Scientific Question Answering](https://mlsanity.com/paper/2511.16283/) (2025)
- [PAR2-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering](https://mlsanity.com/paper/2603.29085/) (2026)
- [IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answering](https://mlsanity.com/paper/2608.13588/) (2026)
- [Dense Process Supervision for Search Agents via Fact Utility Estimation](https://mlsanity.com/paper/2609.00833/) (2026)
- [MAB-DQA: Addressing Query Aspect Importance in Document Question Answering with Multi-Armed Bandits](https://mlsanity.com/paper/2604.08952/) (2026)
- [SEARCH-R: Structured Entity-Aware Retrieval with Chain-of-Reasoning Navigator for Multi-hop Question Answering](https://mlsanity.com/paper/2604.24515/) (2026)
- [D2F-ReAG: Dynamic Decomposition and Filtering for Multi-Hop Reasoning-Augmented Generation](https://mlsanity.com/paper/2608.04444/) (2026)
- [HyperProve: Answer-Guided Hypergraph Expansion for Multi-Hop Question Answering](https://mlsanity.com/paper/2609.13768/) (2026)
- [Beyond Similarity through Zero-Token Geometric Graphs for Multi-Hop RAG](https://mlsanity.com/paper/2609.19622/) (2026)
- [ANTMAN: Adaptive Need Tracking for Multi-Agent Navigation in Large Information Spaces](https://mlsanity.com/paper/2609.33326/) (2026)

## Abstract (arXiv)

In this paper, we identify a critical problem, "lost-in-retrieval", in retrieval-augmented multi-hop question answering (QA): the key entities are missed in LLMs' sub-question decomposition. "Lost-in-retrieval" significantly degrades the retrieval performance, which disrupts the reasoning chain and leads to the incorrect answers. To resolve this problem, we propose a progressive retrieval and rewriting method, namely ChainRAG, which sequentially handles each sub-question by completing missing key entities and retrieving relevant sentences from a sentence graph for answer generation. Each step in our retrieval and rewriting process builds upon the previous one, creating a seamless chain that leads to accurate retrieval and answers. Finally, all retrieved sentences and sub-question answers are integrated to generate a comprehensive answer to the original question. We evaluate ChainRAG on three multi-hop QA datasets - MuSiQue, 2Wiki, and HotpotQA - using three large language models: GPT4o-mini, Qwen2.5-72B, and GLM-4-Plus. Empirical results demonstrate that ChainRAG consistently outperforms baselines in both effectiveness and efficiency.