mlsanityData to 7 Oct 2026

MuSiQue: Multihop Questions via Single-hop Question Composition

arXiv 2108.00573 ↗First posted 2 Aug 2021cs.CL cs.AIcited by 893

Not on the map. Why?

Cited by 267 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Multihop reasoning remains an elusive goal as existing multihop benchmarks are known to be largely solvable via shortcuts. Can we create a question answering (QA) dataset that, by construction, requires proper multihop reasoning? To this end, we introduce a bottom-up approach that systematically selects composable pairs of single-hop questions that are connected, i.e., where one reasoning step critically relies on information from another. This bottom-up methodology lets us explore a vast space of questions and add stringent filters as well as other mechanisms targeting connected reasoning. It provides fine-grained control over the construction process and the properties of the resulting k-hop questions. We use this methodology to create MuSiQue-Ans, a new multihop QA dataset with 25K 2-4 hop questions. Relative to existing datasets, MuSiQue-Ans is more difficult overall (3x increase in human-machine gap), and harder to cheat via disconnected reasoning (e.g., a single-hop model has a 30 point drop in F1). We further add unanswerable contrast questions to produce a more stringent dataset, MuSiQue-Full. We hope our datasets will help the NLP community develop models that perform genuine multihop reasoning.

Key facts. Cited by 893 papers in the map (data through 7 Oct 2026). Builds on 43 of its 57 references that are in the map. First posted on arXiv 2 Aug 2021.

Builds on

43 of 57 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2020Longformer: The Long-Document Transformercited by 3,362
  9. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  10. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  11. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  12. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
31 more
  1. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  2. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  3. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  4. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  5. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  6. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  7. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  8. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  9. 2018AllenNLP: A Deep Semantic Natural Language Processing Platformcited by 347
  10. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  11. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  12. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  13. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 199
  14. 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
  15. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  16. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 137
  17. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  18. 2019Multi-hop Reading Comprehension through Question Decomposition and Rescoringcited by 119
  19. 2019Compositional Questions Do Not Necessitate Multi-hop Reasoningcited by 87
  20. 2018How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarkscited by 86
  21. 2019Scalable Zero-shot Entity Linking with Dense Entity Retrievalcited by 85
  22. 2020Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Modelscited by 65
  23. 2018WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-Hop Inferencecited by 60
  24. 2019Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documentscited by 57
  25. 2020IIRC: A Dataset of Incomplete Information Reading Comprehension Questionscited by 52
  26. 2019Understanding Dataset Design Choices for Multi-hop Reasoningcited by 50
  27. 2020Answering Open-Domain Questions of Varying Reasoning Steps from Textcited by 43
  28. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
  29. 2020Unsupervised Multi-hop Question Answering by Question Generationcited by 38
  30. 2020Evaluating Models' Local Decision Boundaries via Contrast Setscited by 24
  31. 2020A Simple Yet Strong Pipeline for HotpotQAcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 12; 2023: 37; 2024: 95; 2025: 292; 2026: 456.

893 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  3. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  4. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  6. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  7. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  8. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  9. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 442
  10. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  11. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
  12. 2025Memory in the Age of AI Agentscited by 285
38 more
  1. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  2. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 268
  3. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
  4. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  5. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  6. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 235
  7. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 232
  8. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  9. 2025From RAG to Memory: Non-Parametric Continual Learning for Large Language Modelscited by 223
  10. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 218
  11. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  12. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  13. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  14. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  15. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
  16. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 180
  17. 2024An Empirical Study of Mamba-based Language Modelscited by 177
  18. 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 175
  19. 2025Agentic Reinforced Policy Optimizationcited by 166
  20. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  21. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
  22. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 157
  23. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 152
  24. 2023Knowledge Graph Prompting for Multi-Document Question Answeringcited by 150
  25. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 144
  26. 2025Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RLcited by 144
  27. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 129
  28. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 129
  29. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  30. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 119
  31. 2024Make Your LLM Fully Utilize the Contextcited by 118
  32. 2024MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentationcited by 118
  33. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  34. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  35. 2023Retrieval meets Long Context Large Language Modelscited by 114
  36. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
  37. 2024AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agentscited by 100
  38. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learningcited by 94

50 most cited of 893 All 893 (JSON)