mlsanityData to 7 Oct 2026

MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries

arXiv 2401.15391 ↗First posted 27 Jan 2024cs.CLcited by 253

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-augmented generation (RAG) augments large language models (LLM) by retrieving relevant knowledge, showing promising potential in mitigating LLM hallucinations and enhancing response quality, thereby facilitating the great adoption of LLMs in practice. However, we find that existing RAG systems are inadequate in answering multi-hop queries, which require retrieving and reasoning over multiple pieces of supporting evidence. Furthermore, to our knowledge, no existing RAG benchmarking dataset focuses on multi-hop queries. In this paper, we develop a novel dataset, MultiHop-RAG, which consists of a knowledge base, a large collection of multi-hop queries, their ground-truth answers, and the associated supporting evidence. We detail the procedure of building the dataset, utilizing an English news article dataset as the underlying RAG knowledge base. We demonstrate the benchmarking utility of MultiHop-RAG in two experiments. The first experiment compares different embedding models for retrieving evidence for multi-hop queries. In the second experiment, we examine the capabilities of various state-of-the-art LLMs, including GPT-4, PaLM, and Llama2-70B, in reasoning and answering multi-hop queries given the evidence. Both experiments reveal that existing RAG methods perform unsatisfactorily in retrieving and answering multi-hop queries. We hope MultiHop-RAG will be a valuable resource for the community in developing effective RAG systems, thereby facilitating greater adoption of LLMs in practice. The MultiHop-RAG and implemented RAG system is publicly available at https://github.com/yixuantt/MultiHop-RAG/.

Key facts. Cited by 253 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 16 of its 23 references that are in the map. First posted on arXiv 27 Jan 2024.

Builds on

16 of 23 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems16 Nov
  2. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 14

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  4. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  5. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  7. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  8. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  9. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  10. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  11. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  12. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 199
2 more
  1. 2023RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledgecited by 51
  2. 2023Evaluating Embedding APIs for Information Retrievalcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 42; 2025: 110; 2026: 101.

253 papers.

Sort:

By area:

  1. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  2. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  3. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
  4. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  5. 2024LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domaincited by 113
  6. 2024CRAG -- Comprehensive RAG Benchmarkcited by 94
  7. 2024GRAG: Graph Retrieval-Augmented Generationcited by 77
  8. 2024From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMscited by 77
  9. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  10. 2025Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Controlcited by 56
  11. 2025SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesiscited by 55
  12. 2025RAG vs. GraphRAG: A Systematic Evaluation and Key Insightscited by 51
38 more
  1. 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agentcited by 46
  2. 2025Retrieval-Augmented Generation with Hierarchical Knowledgecited by 46
  3. 2025When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generationcited by 46
  4. 2025In-depth Analysis of Graph-based RAG in a Unified Frameworkcited by 45
  5. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answeringcited by 43
  6. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 40
  7. 2024RAGEval: Scenario Specific RAG Evaluation Dataset Generation Frameworkcited by 39
  8. 2025KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakescited by 30
  9. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 30
  10. 2025ArchRAG: Attributed Community-based Hierarchical Retrieval-Augmented Generationcited by 27
  11. 2024Cofca: A Step-Wise Counterfactual Multi-hop QA benchmarkcited by 26
  12. 2025NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodescited by 25
  13. 2024OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generationcited by 24
  14. 2025Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicingcited by 24
  15. 2025ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learningcited by 24
  16. 2024Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Basescited by 23
  17. 2024SiReRAG: Indexing Similar and Related Information for Multihop Reasoningcited by 23
  18. 2024OG-RAG: Ontology-Grounded Retrieval-Augmented Generation For Large Language Modelscited by 23
  19. 2025Benchmarking Deep Search over Heterogeneous Enterprise Datacited by 23
  20. 2025CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehensioncited by 23
  21. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 22
  22. 2026ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarioscited by 22
  23. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 21
  24. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  25. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  26. 2025GaRAGe: A Benchmark with Grounding Annotations for RAG Evaluationcited by 18
  27. 2024Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teachingcited by 17
  28. 2024SEC-QA: A Systematic Evaluation Corpus for Financial QAcited by 17
  29. 2024Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perceptioncited by 17
  30. 2024Multi-Meta-RAG: Improving RAG for Multi-Hop Queries using Database Filtering with LLM-Extracted Metadatacited by 16
  31. 2025Explicit v.s. Implicit Memory: Exploring Multi-hop Complex Reasoning Over Personalized Informationcited by 16
  32. 2025CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answeringcited by 15
  33. 2025Graph RAG-Tool Fusioncited by 15
  34. 2025InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented Generationcited by 15
  35. 2024PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Samplingcited by 14
  36. 2025PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluationcited by 14
  37. 2025EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corporacited by 14
  38. 2025Vendi-RAG: Adaptively Trading-Off Diversity And Quality Significantly Improves Retrieval Augmented Generation With LLMscited by 13

50 most cited of 253 All 253 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  2. 2026ConRAG: Consensus-Driven Multi-View Retrieval for Multi-Hop Question Answering
  3. 2026When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answering
  4. 2026LIT-RAGBench: Benchmarking Generator Capabilities of Large Language Models in Retrieval-Augmented Generation
  5. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  6. 2025Multi-hop Reasoning via Early Knowledge Alignment
  7. 2025Optimizing Multi-Hop Document Retrieval Through Intermediate Representations
  8. 2025RAGtifier: Evaluating RAG Generation Approaches of State-of-the-Art RAG Systems for the SIGIR LiveRAG Competition