mlsanityData to 7 Oct 2026

MCScript2.0: A Machine Comprehension Corpus Focused on Script Events and Participants

arXiv 1905.09531 ↗First posted 23 May 2019cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

We introduce MCScript2.0, a machine comprehension corpus for the end-to-end evaluation of script knowledge. MCScript2.0 contains approx. 20,000 questions on approx. 3,500 texts, crowdsourced based on a new collection process that results in challenging questions. Half of the questions cannot be answered from the reading texts, but require the use of commonsense and, in particular, script knowledge. We give a thorough analysis of our corpus and show that while the task is not challenging to humans, existing machine comprehension models fail to perform well on the data, even if they make use of a commonsense knowledge base. The dataset is available at http://www.sfb1102.uni-saarland.de/?page_id=2582

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 10; 2022: 5; 2023: 4; 2024: 1.

20 papers.

Sort:

By area:

  1. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  2. 2022UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Trainingcited by 57
  3. 2021English Machine Reading Comprehension Datasets: A Surveycited by 26
  4. 2021Combining pre-trained language models and structured knowledgecited by 25
  5. 2021What do Large Language Models Learn about Scripts?cited by 19
  6. 2022RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Questioncited by 17
  7. 2021Salience-Aware Event Chain Modeling for Narrative Understandingcited by 14
  8. 2021Identify, Align, and Integrate: Matching Knowledge Graphs to Commonsense Reasoning Taskscited by 9
  9. 2023Long-Tailed Question Answering in an Open Worldcited by 7
  10. 2021StoryDB: Broad Multi-language Narrative Datasetcited by 5
  11. 2021DYPLODOC: Dynamic Plots for Document Classificationcited by 4
  12. 2022A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehensioncited by 4
8 more
  1. 2022What is Wrong with Language Models that Can Not Tell a Story?cited by 4
  2. 2021Commonsense Knowledge in Word Associations and ConceptNetcited by 2
  3. 2021Actionable Entities Recognition Benchmark for Interactive Fictioncited by 2
  4. 2023Analyzing the Effectiveness of the Underlying Reasoning Tasks in Multi-hop Question Answeringcited by 2
  5. 2023DISTO: Evaluating Textual Distractors for Multi-Choice Questions using Negative Sampling based Approachcited by 1
  6. 2021Understanding in Artificial Intelligencecited by 0
  7. 2022Using contradictions improves question answering systemscited by 0
  8. 2024Recent Advances in Multi-Choice Machine Reading Comprehension: A Survey on Methods and Datasetscited by 0