# A Comprehensive Survey on Multi-hop Machine Reading Comprehension Datasets and Metrics

arXiv: https://arxiv.org/abs/2212.04070 · HTML: https://mlsanity.com/paper/2212.04070/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 8 Dec 2022.

## Builds on (most cited first, 30 of 34)

- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [QASC: A Dataset for Question Answering via Sentence Composition](https://mlsanity.com/paper/1910.11473/) (2019)
- [Constructing Datasets for Multi-hop Reading Comprehension Across Documents](https://mlsanity.com/paper/1710.06481/) (2017)
- [Hierarchical Graph Network for Multi-hop Question Answering](https://mlsanity.com/paper/1911.03631/) (2019)
- [A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task](https://mlsanity.com/paper/1606.02858/) (2016)
- [Multi-hop Question Answering via Reasoning Chains](https://mlsanity.com/paper/1910.02610/) (2019)
- [A Survey on Machine Reading Comprehension Systems](https://mlsanity.com/paper/2001.01582/) (2020)
- [A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets](https://mlsanity.com/paper/2006.11880/) (2020)
- [Gated-Attention Readers for Text Comprehension](https://mlsanity.com/paper/1606.01549/) (2016)
- [R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reason](https://mlsanity.com/paper/1910.04601/) (2019)
- [A Survey on Explainability in Machine Reading Comprehension](https://mlsanity.com/paper/2010.00389/) (2020)
- [Attention-over-Attention Neural Networks for Reading Comprehension](https://mlsanity.com/paper/1607.04423/) (2016)
- [Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoning](https://mlsanity.com/paper/1909.05803/) (2019)
- [Efficient and Robust Question Answering from Minimal Context over Documents](https://mlsanity.com/paper/1805.08092/) (2018)
- [ReasoNet: Learning to Stop Reading in Machine Comprehension](https://mlsanity.com/paper/1609.05284/) (2016)
- [Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond](https://mlsanity.com/paper/2005.06249/) (2020)
- [Exploiting Explicit Paths for Multi-hop Reading Comprehension](https://mlsanity.com/paper/1811.01127/) (2018)
- [Question Answering via Integer Programming over Semi-Structured Knowledge](https://mlsanity.com/paper/1604.06076/) (2016)
- [Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy](https://mlsanity.com/paper/2107.11823/) (2021)

## Built on by (most cited first, 1 of 1)

- [Interpretable Physics Reasoning and Performance Taxonomy in Vision-Language Models](https://mlsanity.com/paper/2509.08270/) (2025)

## Abstract (arXiv)

Multi-hop Machine reading comprehension is a challenging task with aim of answering a question based on disjoint pieces of information across the different passages. The evaluation metrics and datasets are a vital part of multi-hop MRC because it is not possible to train and evaluate models without them, also, the proposed challenges by datasets often are an important motivation for improving the existing models. Due to increasing attention to this field, it is necessary and worth reviewing them in detail. This study aims to present a comprehensive survey on recent advances in multi-hop MRC evaluation metrics and datasets. In this regard, first, the multi-hop MRC problem definition will be presented, then the evaluation metrics based on their multi-hop aspect will be investigated. Also, 15 multi-hop datasets have been reviewed in detail from 2017 to 2022, and a comprehensive analysis has been prepared at the end. Finally, open issues in this field have been discussed.