# Compositional Questions Do Not Necessitate Multi-hop Reasoning

arXiv: https://arxiv.org/abs/1906.02900 · HTML: https://mlsanity.com/paper/1906.02900/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2019.

## Built on by (most cited first, 30 of 88)

- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [QuALITY: Question Answering with Long Input Texts, Yes!](https://mlsanity.com/paper/2112.08608/) (2021)
- [MultiModalQA: Complex Question Answering over Text, Tables and Images](https://mlsanity.com/paper/2104.06039/) (2021)
- [Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models](https://mlsanity.com/paper/2402.14848/) (2024)
- [KLUE: Korean Language Understanding Evaluation](https://mlsanity.com/paper/2105.09680/) (2021)
- [CommonsenseQA 2.0: Exposing the Limits of AI through Gamification](https://mlsanity.com/paper/2201.05320/) (2022)
- [GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective](https://mlsanity.com/paper/2211.08073/) (2022)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [Joint Passage Ranking for Diverse Multi-Answer Retrieval](https://mlsanity.com/paper/2104.08445/) (2021)
- [Why Machine Reading Comprehension Models Learn Shortcuts?](https://mlsanity.com/paper/2106.01024/) (2021)
- [MoreHopQA: More Than Multi-hop Reasoning](https://mlsanity.com/paper/2406.13397/) (2024)
- [Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answering](https://mlsanity.com/paper/2103.05568/) (2021)
- [FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Models](https://mlsanity.com/paper/2402.14116/) (2024)
- [Flexible Generation of Natural Language Deductions](https://mlsanity.com/paper/2104.08825/) (2021)
- [Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?](https://mlsanity.com/paper/2411.16679/) (2024)
- [Analyzing Dynamic Adversarial Training Data in the Limit](https://mlsanity.com/paper/2110.08514/) (2021)
- [Rissanen Data Analysis: Examining Dataset Characteristics via Description Length](https://mlsanity.com/paper/2103.03872/) (2021)
- [Reasoning over Public and Private Data in Retrieval-Based Systems](https://mlsanity.com/paper/2203.11027/) (2022)
- [Do Question Answering Modeling Improvements Hold Across Benchmarks?](https://mlsanity.com/paper/2102.01065/) (2021)
- [Connecting Attributions and QA Model Behavior on Realistic Counterfactuals](https://mlsanity.com/paper/2104.04515/) (2021)
- [MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge](https://mlsanity.com/paper/2412.17032/) (2024)
- [Saving Dense Retriever from Shortcut Dependency in Conversational Search](https://mlsanity.com/paper/2202.07280/) (2022)
- [Uncertainty Calibration for Ensemble-Based Debiasing Methods](https://mlsanity.com/paper/2111.04104/) (2021)
- [Complex Factoid Question Answering with a Free-Text Knowledge Graph](https://mlsanity.com/paper/2103.12876/) (2021)
- [NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset](https://mlsanity.com/paper/2109.10604/) (2021)
- [Hierarchical Retrieval-Augmented Generation Model with Rethink for Multi-hop Question Answering](https://mlsanity.com/paper/2408.11875/) (2024)
- [FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Data](https://mlsanity.com/paper/2501.17144/) (2025)
- [Robustifying Multi-hop QA through Pseudo-Evidentiality Training](https://mlsanity.com/paper/2107.03242/) (2021)
- [Hey AI, Can You Solve Complex Tasks by Talking to Agents?](https://mlsanity.com/paper/2110.08542/) (2021)

## Abstract (arXiv)

Multi-hop reading comprehension (RC) questions are challenging because they require reading and reasoning over multiple paragraphs. We argue that it can be difficult to construct large multi-hop RC datasets. For example, even highly compositional questions can be answered with a single hop if they target specific entity types, or the facts needed to answer them are redundant. Our analysis is centered on HotpotQA, where we show that single-hop reasoning can solve much more of the dataset than previously thought. We introduce a single-hop BERT-based RC model that achieves 67 F1---comparable to state-of-the-art multi-hop models. We also design an evaluation setting where humans are not shown all of the necessary paragraphs for the intended multi-hop reasoning but can still answer over 80% of questions. Together with detailed error analysis, these results suggest there should be an increasing focus on the role of evidence in multi-hop reasoning and possibly even a shift towards information retrieval style evaluations with large and diverse evidence collections.