mlsanityData to 7 Oct 2026

A Comprehensive Survey on Multi-hop Machine Reading Comprehension Datasets and Metrics

arXiv 2212.04070 ↗First posted 8 Dec 2022cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Multi-hop Machine reading comprehension is a challenging task with aim of answering a question based on disjoint pieces of information across the different passages. The evaluation metrics and datasets are a vital part of multi-hop MRC because it is not possible to train and evaluate models without them, also, the proposed challenges by datasets often are an important motivation for improving the existing models. Due to increasing attention to this field, it is necessary and worth reviewing them in detail. This study aims to present a comprehensive survey on recent advances in multi-hop MRC evaluation metrics and datasets. In this regard, first, the multi-hop MRC problem definition will be presented, then the evaluation metrics based on their multi-hop aspect will be investigated. Also, 15 multi-hop datasets have been reviewed in detail from 2017 to 2022, and a comprehensive analysis has been prepared at the end. Finally, open issues in this field have been discussed.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 8 Dec 2022.

Builds on

34 of 45 references.

References · 34

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  5. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  6. 2015Teaching Machines to Read and Comprehendcited by 1,039
  7. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  8. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  9. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  10. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  11. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  12. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
22 more
  1. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  2. 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
  3. 2019Hierarchical Graph Network for Multi-hop Question Answeringcited by 83
  4. 2016A Thorough Examination of the CNN/Daily Mail Reading Comprehension Taskcited by 71
  5. 2019Multi-hop Question Answering via Reasoning Chainscited by 42
  6. 2020A Survey on Machine Reading Comprehension Systemscited by 41
  7. 2020A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasetscited by 40
  8. 2016Gated-Attention Readers for Text Comprehensioncited by 37
  9. 2019R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reasoncited by 36
  10. 2020A Survey on Explainability in Machine Reading Comprehensioncited by 33
  11. 2016Attention-over-Attention Neural Networks for Reading Comprehensioncited by 31
  12. 2019Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoningcited by 28
  13. 2018Efficient and Robust Question Answering from Minimal Context over Documentscited by 26
  14. 2016ReasoNet: Learning to Stop Reading in Machine Comprehensioncited by 21
  15. 2020Machine Reading Comprehension: The Role of Contextualized Language Models and Beyondcited by 21
  16. 2018Exploiting Explicit Paths for Multi-hop Reading Comprehensioncited by 16
  17. 2016Question Answering via Integer Programming over Semi-Structured Knowledgecited by 13
  18. 2021Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategycited by 13
  19. 2019Explore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehensioncited by 12
  20. 2021More Than Reading Comprehension: A Survey on Datasets and Metrics of Textual Question Answeringcited by 8
  21. 2021ComQA:Compositional Question Answering via Hierarchical Graph Neural Networkscited by 4
  22. 2022Semantic Sentence Composition Reasoning for Multi-Hop Question Answeringcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Interpretable Physics Reasoning and Performance Taxonomy in Vision-Language Modelscited by 0