Data to 14 Oct 2026

Compositional Questions Do Not Necessitate Multi-hop Reasoning

arXiv 1906.02900 ↗First posted 7 Jun 2019cs.CL cs.AIcited by 88

Not on the map. Why?

Cited by 21 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Multi-hop reading comprehension (RC) questions are challenging because they require reading and reasoning over multiple paragraphs. We argue that it can be difficult to construct large multi-hop RC datasets. For example, even highly compositional questions can be answered with a single hop if they target specific entity types, or the facts needed to answer them are redundant. Our analysis is centered on HotpotQA, where we show that single-hop reasoning can solve much more of the dataset than previously thought. We introduce a single-hop BERT-based RC model that achieves 67 F1---comparable to state-of-the-art multi-hop models. We also design an evaluation setting where humans are not shown all of the necessary paragraphs for the intended multi-hop reasoning but can still answer over 80% of questions. Together with detailed error analysis, these results suggest there should be an increasing focus on the role of evidence in multi-hop reasoning and possibly even a shift towards information retrieval style evaluations with large and diverse evidence collections.

Key facts. Cited by 88 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 30; 2022: 19; 2023: 6; 2024: 13; 2025: 10; 2026: 10.

88 papers.

Sort:

By area:

2023Large Language Models are not Fair Evaluatorscited by 1,085
2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 208
2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 179
2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 152
2021KLUE: Korean Language Understanding Evaluationcited by 114
2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 109
2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
2023Natural Language Reasoning, A Surveycited by 61
2021Joint Passage Ranking for Diverse Multi-Answer Retrievalcited by 43
2021Why Machine Reading Comprehension Models Learn Shortcuts?cited by 40
2024MoreHopQA: More Than Multi-hop Reasoningcited by 37

38 more

2021Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answeringcited by 33
2024FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Modelscited by 29
2021Flexible Generation of Natural Language Deductionscited by 28
2024Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?cited by 28
2021Analyzing Dynamic Adversarial Training Data in the Limitcited by 24
2021Rissanen Data Analysis: Examining Dataset Characteristics via Description Lengthcited by 22
2022Reasoning over Public and Private Data in Retrieval-Based Systemscited by 21
2021Do Question Answering Modeling Improvements Hold Across Benchmarks?cited by 20
2021Connecting Attributions and QA Model Behavior on Realistic Counterfactualscited by 20
2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledgecited by 20
2022Saving Dense Retriever from Shortcut Dependency in Conversational Searchcited by 19
2021Uncertainty Calibration for Ensemble-Based Debiasing Methodscited by 16
2021Complex Factoid Question Answering with a Free-Text Knowledge Graphcited by 14
2021NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Datasetcited by 14
2024Hierarchical Retrieval-Augmented Generation Model with Rethink for Multi-hop Question Answeringcited by 14
2025FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Datacited by 14
2021Robustifying Multi-hop QA through Pseudo-Evidentiality Trainingcited by 13
2021Hey AI, Can You Solve Complex Tasks by Talking to Agents?cited by 13
2022Machine Reading, Fast and Slow: When Do Models "Understand" Language?cited by 13
2021Do Natural Language Explanations Represent Valid Logical Arguments? Verifying Entailment in Explainable NLI Gold Standardscited by 12
2022From Easy to Hard: Two-stage Selector and Reader for Multi-hop Question Answeringcited by 12
2022What Makes Reading Comprehension Questions Difficult?cited by 11
2021Mitigating the Position Bias of Transformer Models in Passage Re-Rankingcited by 10
2024Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers?cited by 10
2022Automated Crossword Solvingcited by 9
2023What Else Do I Need to Know? The Effect of Background Information on Users' Reliance on QA Systemscited by 9
2021Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classificationcited by 8
2025GRADE: Generating multi-hop QA and fine-gRAined Difficulty matrix for RAG Evaluationcited by 8
2021General Greedy De-bias Learningcited by 7
2022Teaching Broad Reasoning Skills for Multi-Step QA by Generating Hard Contextscited by 7
2022Understanding and Improving Zero-shot Multi-hop Reasoning in Generative Question Answeringcited by 7
2022Counterfactual Multihop QA: A Cause-Effect Approach for Reducing Disconnected Reasoningcited by 7
2021Paired Examples as Indirect Supervision in Latent Decision Modelscited by 6
2021Grounded Graph Decoding Improves Compositional Generalization in Question Answeringcited by 6
2024FSM: A Finite State Machine Based Zero-Shot Prompting Paradigm for Multi-Hop Question Answeringcited by 6
2021Distantly-Supervised Evidence Retrieval Enables Question Answering without Evidence Annotationcited by 5
2023HOP, UNION, GENERATE: Explainable Multi-hop Reasoning without Rationale Supervisioncited by 5
2025Reward Hacking Mitigation using Verifiable Composite Rewardscited by 5

50 most cited of 88 All 88 (JSON)