# A Simple Yet Strong Pipeline for HotpotQA

arXiv: https://arxiv.org/abs/2004.06753 · HTML: https://mlsanity.com/paper/2004.06753/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2020.

## Built on by (most cited first, 24 of 24)

- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation](https://mlsanity.com/paper/2409.13731/) (2024)
- [Connecting Attributions and QA Model Behavior on Realistic Counterfactuals](https://mlsanity.com/paper/2104.04515/) (2021)
- [Robustifying Multi-hop QA through Pseudo-Evidentiality Training](https://mlsanity.com/paper/2107.03242/) (2021)
- [Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy](https://mlsanity.com/paper/2107.11823/) (2021)
- [From Easy to Hard: Two-stage Selector and Reader for Multi-hop Question Answering](https://mlsanity.com/paper/2205.11729/) (2022)
- [CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoning](https://mlsanity.com/paper/2502.02390/) (2025)
- [Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalization](https://mlsanity.com/paper/2103.12235/) (2021)
- [Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension](https://mlsanity.com/paper/2109.06853/) (2021)
- [Chain-of-Questions Training with Latent Answers for Robust Multistep Question Answering](https://mlsanity.com/paper/2305.14901/) (2023)
- [Counterfactual Multihop QA: A Cause-Effect Approach for Reducing Disconnected Reasoning](https://mlsanity.com/paper/2210.07138/) (2022)
- [Beyond the Granularity: Multi-Perspective Dialogue Collaborative Selection for Dialogue State Tracking](https://mlsanity.com/paper/2205.10059/) (2022)
- [A Survey on non-English Question Answering Dataset](https://mlsanity.com/paper/2112.13634/) (2021)
- [HOP, UNION, GENERATE: Explainable Multi-hop Reasoning without Rationale Supervision](https://mlsanity.com/paper/2305.14237/) (2023)
- [A Simple Approach to Jointly Rank Passages and Select Relevant Sentences in the OBQA Context](https://mlsanity.com/paper/2109.10497/) (2021)
- [Rethinking Label Smoothing on Multi-hop Question Answering](https://mlsanity.com/paper/2212.09512/) (2022)
- [Classification of Goods Using Text Descriptions With Sentences Retrieval](https://mlsanity.com/paper/2111.01663/) (2021)
- [Towards Interpretable and Reliable Reading Comprehension: A Pipeline Model with Unanswerability Prediction](https://mlsanity.com/paper/2111.09029/) (2021)
- [QA4QG: Using Question Answering to Constrain Multi-Hop Question Generation](https://mlsanity.com/paper/2202.06538/) (2022)
- [Better Retrieval May Not Lead to Better Question Answering](https://mlsanity.com/paper/2205.03685/) (2022)
- [Explainable Product Classification for Customs](https://mlsanity.com/paper/2311.10922/) (2023)
- [Uncertainty Guided Global Memory Improves Multi-Hop Question Answering](https://mlsanity.com/paper/2311.18151/) (2023)
- [Relational Graph Convolutional Neural Networks for Multihop Reasoning: A Comparative Study](https://mlsanity.com/paper/2210.06418/) (2022)
- [Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks](https://mlsanity.com/paper/2501.06286/) (2025)

## Abstract (arXiv)

State-of-the-art models for multi-hop question answering typically augment large-scale language models like BERT with additional, intuitively useful capabilities such as named entity recognition, graph-based reasoning, and question decomposition. However, does their strong performance on popular multi-hop datasets really justify this added design complexity? Our results suggest that the answer may be no, because even our simple pipeline based on BERT, named Quark, performs surprisingly well. Specifically, on HotpotQA, Quark outperforms these models on both question answering and support identification (and achieves performance very close to a RoBERTa model). Our pipeline has three steps: 1) use BERT to identify potentially relevant sentences independently of each other; 2) feed the set of selected sentences as context into a standard BERT span prediction model to choose an answer; and 3) use the sentence selection model, now with the chosen answer, to produce supporting sentences. The strong performance of Quark resurfaces the importance of carefully exploring simple model designs before using popular benchmarks to justify the value of complex techniques.