# MCScript2.0: A Machine Comprehension Corpus Focused on Script Events and Participants

arXiv: https://arxiv.org/abs/1905.09531 · HTML: https://mlsanity.com/paper/1905.09531/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2019.

## Built on by (most cited first, 20 of 20)

- [The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants](https://mlsanity.com/paper/2308.16884/) (2023)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [English Machine Reading Comprehension Datasets: A Survey](https://mlsanity.com/paper/2101.10421/) (2021)
- [Combining pre-trained language models and structured knowledge](https://mlsanity.com/paper/2101.12294/) (2021)
- [What do Large Language Models Learn about Scripts?](https://mlsanity.com/paper/2112.13834/) (2021)
- [RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question](https://mlsanity.com/paper/2211.01482/) (2022)
- [Salience-Aware Event Chain Modeling for Narrative Understanding](https://mlsanity.com/paper/2109.10475/) (2021)
- [Identify, Align, and Integrate: Matching Knowledge Graphs to Commonsense Reasoning Tasks](https://mlsanity.com/paper/2104.10193/) (2021)
- [Long-Tailed Question Answering in an Open World](https://mlsanity.com/paper/2305.06557/) (2023)
- [StoryDB: Broad Multi-language Narrative Dataset](https://mlsanity.com/paper/2109.14396/) (2021)
- [DYPLODOC: Dynamic Plots for Document Classification](https://mlsanity.com/paper/2107.12226/) (2021)
- [A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehension](https://mlsanity.com/paper/2209.01824/) (2022)
- [What is Wrong with Language Models that Can Not Tell a Story?](https://mlsanity.com/paper/2211.05044/) (2022)
- [Commonsense Knowledge in Word Associations and ConceptNet](https://mlsanity.com/paper/2109.09309/) (2021)
- [Actionable Entities Recognition Benchmark for Interactive Fiction](https://mlsanity.com/paper/2109.13855/) (2021)
- [Analyzing the Effectiveness of the Underlying Reasoning Tasks in Multi-hop Question Answering](https://mlsanity.com/paper/2302.05963/) (2023)
- [DISTO: Evaluating Textual Distractors for Multi-Choice Questions using Negative Sampling based Approach](https://mlsanity.com/paper/2304.04881/) (2023)
- [Understanding in Artificial Intelligence](https://mlsanity.com/paper/2101.06573/) (2021)
- [Using contradictions improves question answering systems](https://mlsanity.com/paper/2211.05598/) (2022)
- [Recent Advances in Multi-Choice Machine Reading Comprehension: A Survey on Methods and Datasets](https://mlsanity.com/paper/2408.02114/) (2024)

## Abstract (arXiv)

We introduce MCScript2.0, a machine comprehension corpus for the end-to-end evaluation of script knowledge. MCScript2.0 contains approx. 20,000 questions on approx. 3,500 texts, crowdsourced based on a new collection process that results in challenging questions. Half of the questions cannot be answered from the reading texts, but require the use of commonsense and, in particular, script knowledge. We give a thorough analysis of our corpus and show that while the task is not challenging to humans, existing machine comprehension models fail to perform well on the data, even if they make use of a commonsense knowledge base. The dataset is available at http://www.sfb1102.uni-saarland.de/?page_id=2582