mlsanityData to 7 Oct 2026

Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps

arXiv 2011.01060 ↗First posted 2 Nov 2020cs.CLcited by 1,259

Not on the map. Why?

Cited by 451 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

A multi-hop question answering (QA) dataset aims to test reasoning and inference skills by requiring a model to read multiple paragraphs to answer a given question. However, current datasets do not provide a complete explanation for the reasoning process from the question to the answer. Further, previous studies revealed that many examples in existing multi-hop datasets do not require multi-hop reasoning to answer a question. In this study, we present a new multi-hop QA dataset, called 2WikiMultiHopQA, which uses structured and unstructured data. In our dataset, we introduce the evidence information containing a reasoning path for multi-hop questions. The evidence information has two benefits: (i) providing a comprehensive explanation for predictions and (ii) evaluating the reasoning skills of a model. We carefully design a pipeline and a set of templates when generating a question-answer pair that guarantees the multi-hop steps and the quality of the questions. We also exploit the structured format in Wikidata and use logical rules to create questions that are natural but still require multi-hop reasoning. Through experiments, we demonstrate that our dataset is challenging for multi-hop models and it ensures that multi-hop reasoning is required.

Key facts. Cited by 1,259 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 14; 2023: 34; 2024: 169; 2025: 438; 2026: 597.

1,259 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  3. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  4. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  6. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  7. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  8. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  9. 2023Active Retrieval Augmented Generationcited by 506
  10. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  11. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 442
  12. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
38 more
  1. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  2. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  3. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  4. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  5. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  6. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
  7. 2025Memory in the Age of AI Agentscited by 285
  8. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  9. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 268
  10. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  11. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 253
  12. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  13. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 242
  14. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 235
  15. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  16. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 226
  17. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 218
  18. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  19. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  20. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  21. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  22. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  23. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 180
  24. 2024An Empirical Study of Mamba-based Language Modelscited by 177
  25. 2025Agentic Reinforced Policy Optimizationcited by 166
  26. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  27. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 152
  28. 2023Knowledge Graph Prompting for Multi-Document Question Answeringcited by 150
  29. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 144
  30. 2025Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RLcited by 144
  31. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  32. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 129
  33. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  34. 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 122
  35. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 119
  36. 2024Make Your LLM Fully Utilize the Contextcited by 118
  37. 2024MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentationcited by 118
  38. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approachcited by 117

50 most cited of 1,259 All 1,259 (JSON)