# MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge

arXiv: https://arxiv.org/abs/1803.05223 · HTML: https://mlsanity.com/paper/1803.05223/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2018.

## Built on by (most cited first, 30 of 30)

- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [Comparing Test Sets with Item Response Theory](https://mlsanity.com/paper/2106.00840/) (2021)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [Benchmarks for Automated Commonsense Reasoning: A Survey](https://mlsanity.com/paper/2302.04752/) (2023)
- [Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanese](https://mlsanity.com/paper/2402.17302/) (2024)
- [Conversational Question Answering: A Survey](https://mlsanity.com/paper/2106.00874/) (2021)
- [Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once?](https://mlsanity.com/paper/2402.11597/) (2024)
- [Automatic Story Generation: Challenges and Attempts](https://mlsanity.com/paper/2102.12634/) (2021)
- [English Machine Reading Comprehension Datasets: A Survey](https://mlsanity.com/paper/2101.10421/) (2021)
- [What do Large Language Models Learn about Scripts?](https://mlsanity.com/paper/2112.13834/) (2021)
- [RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question](https://mlsanity.com/paper/2211.01482/) (2022)
- [Prompting with Pseudo-Code Instructions](https://mlsanity.com/paper/2305.11790/) (2023)
- [CALM : A Multi-task Benchmark for Comprehensive Assessment of Language Model Bias](https://mlsanity.com/paper/2308.12539/) (2023)
- [A Survey of Machine Narrative Reading Comprehension Assessments](https://mlsanity.com/paper/2205.00299/) (2022)
- [mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans](https://mlsanity.com/paper/2406.04215/) (2024)
- [Language Model Augmented Relevance Score](https://mlsanity.com/paper/2108.08485/) (2021)
- [Long-Tailed Question Answering in an Open World](https://mlsanity.com/paper/2305.06557/) (2023)
- [Probing Script Knowledge from Pre-Trained Models](https://mlsanity.com/paper/2204.10176/) (2022)
- [A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehension](https://mlsanity.com/paper/2209.01824/) (2022)
- [Encoding Explanatory Knowledge for Zero-shot Science Question Answering](https://mlsanity.com/paper/2105.05737/) (2021)
- [Commonsense Knowledge in Word Associations and ConceptNet](https://mlsanity.com/paper/2109.09309/) (2021)
- [ScriptWorld: Text Based Environment For Learning Procedural Knowledge](https://mlsanity.com/paper/2307.03906/) (2023)
- [Cross-Asset Risk Management: Integrating LLMs for Real-Time Monitoring of Equity, Fixed Income, and Currency Markets](https://mlsanity.com/paper/2504.04292/) (2025)
- [LFED: A Literary Fiction Evaluation Dataset for Large Language Models](https://mlsanity.com/paper/2405.10166/) (2024)
- [ReCAM@IITK at SemEval-2021 Task 4: BERT and ALBERT based Ensemble for Abstract Word Prediction](https://mlsanity.com/paper/2104.01563/) (2021)
- [ListReader: Extracting List-form Answers for Opinion Questions](https://mlsanity.com/paper/2110.11692/) (2021)
- [Using contradictions improves question answering systems](https://mlsanity.com/paper/2211.05598/) (2022)
- [Teach model to answer questions after comprehending the document](https://mlsanity.com/paper/2307.08931/) (2023)
- [Recent Advances in Multi-Choice Machine Reading Comprehension: A Survey on Methods and Datasets](https://mlsanity.com/paper/2408.02114/) (2024)
- [Adaptive Learning Systems: Personalized Curriculum Design Using LLM-Powered Analytics](https://mlsanity.com/paper/2507.18949/) (2025)

## Abstract (arXiv)

We introduce a large dataset of narrative texts and questions about these texts, intended to be used in a machine comprehension task that requires reasoning using commonsense knowledge. Our dataset complements similar datasets in that we focus on stories about everyday activities, such as going to the movies or working in the garden, and that the questions require commonsense knowledge, or more specifically, script knowledge, to be answered. We show that our mode of data collection via crowdsourcing results in a substantial amount of such inference questions. The dataset forms the basis of a shared task on commonsense and script knowledge organized at SemEval 2018 and provides challenging test cases for the broader natural language understanding community.