# Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond

arXiv: https://arxiv.org/abs/2005.06249 · HTML: https://mlsanity.com/paper/2005.06249/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2020.

## Built on by (most cited first, 21 of 21)

- [In-context Pretraining: Language Modeling Beyond Document Boundaries](https://mlsanity.com/paper/2310.10638/) (2023)
- [Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages](https://mlsanity.com/paper/2203.14139/) (2022)
- [Relation-aware Video Reading Comprehension for Temporal Language Grounding](https://mlsanity.com/paper/2110.05717/) (2021)
- [Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-Centric Summarization](https://mlsanity.com/paper/2203.14187/) (2022)
- [Dependency Parsing as MRC-based Span-Span Prediction](https://mlsanity.com/paper/2105.07654/) (2021)
- [Advances in Multi-turn Dialogue Comprehension: A Survey](https://mlsanity.com/paper/2103.03125/) (2021)
- [Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge](https://mlsanity.com/paper/2102.05474/) (2021)
- [Deep Lexical Hypothesis: Identifying personality structure in natural language](https://mlsanity.com/paper/2203.02092/) (2022)
- [QAScore -- An Unsupervised Unreferenced Metric for the Question Generation Evaluation](https://mlsanity.com/paper/2210.04320/) (2022)
- [Multi-tasking Dialogue Comprehension with Discourse Parsing](https://mlsanity.com/paper/2110.03269/) (2021)
- [ChatGPT Incorrectness Detection in Software Reviews](https://mlsanity.com/paper/2403.16347/) (2024)
- [Fine-tuning Transformer-based Encoder for Turkish Language Understanding Tasks](https://mlsanity.com/paper/2401.17396/) (2024)
- [Extending the Scope of Out-of-Domain: Examining QA models in multiple subdomains](https://mlsanity.com/paper/2204.04534/) (2022)
- [A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets](https://mlsanity.com/paper/2304.00483/) (2023)
- [A Comprehensive Survey on Multi-hop Machine Reading Comprehension Datasets and Metrics](https://mlsanity.com/paper/2212.04070/) (2022)
- [A Comprehensive Survey on Multi-hop Machine Reading Comprehension Approaches](https://mlsanity.com/paper/2212.04072/) (2022)
- [Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading Comprehension](https://mlsanity.com/paper/2306.12069/) (2023)
- [Multi-grained Evidence Inference for Multi-choice Reading Comprehension](https://mlsanity.com/paper/2310.18070/) (2023)
- [MRCBert: A Machine Reading ComprehensionApproach for Unsupervised Summarization](https://mlsanity.com/paper/2105.00239/) (2021)
- [Advances in Multi-turn Dialogue Comprehension: A Survey](https://mlsanity.com/paper/2110.04984/) (2021)
- [Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension](https://mlsanity.com/paper/2301.03953/) (2023)

## Abstract (arXiv)

Machine reading comprehension (MRC) aims to teach machines to read and comprehend human languages, which is a long-standing goal of natural language processing (NLP). With the burst of deep neural networks and the evolution of contextualized language models (CLMs), the research of MRC has experienced two significant breakthroughs. MRC and CLM, as a phenomenon, have a great impact on the NLP community. In this survey, we provide a comprehensive and comparative review on MRC covering overall research topics about 1) the origin and development of MRC and CLM, with a particular focus on the role of CLMs; 2) the impact of MRC and CLM to the NLP community; 3) the definition, datasets, and evaluation of MRC; 4) general MRC architecture and technical methods in the view of two-stage Encoder-Decoder solving architecture from the insights of the cognitive process of humans; 5) previous highlights, emerging topics, and our empirical analysis, among which we especially focus on what works in different periods of MRC researches. We propose a full-view categorization and new taxonomies on these topics. The primary views we have arrived at are that 1) MRC boosts the progress from language processing to understanding; 2) the rapid improvement of MRC systems greatly benefits from the development of CLMs; 3) the theme of MRC is gradually moving from shallow text matching to cognitive reasoning.