# A Survey of Machine Narrative Reading Comprehension Assessments

arXiv: https://arxiv.org/abs/2205.00299 · HTML: https://mlsanity.com/paper/2205.00299/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 71 references that are in the map.
- First posted on arXiv 30 Apr 2022.

## Builds on (most cited first, 26 of 26)

- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [BookSum: A Collection of Datasets for Long-form Narrative Summarization](https://mlsanity.com/paper/2105.08209/) (2021)
- [The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations](https://mlsanity.com/paper/1511.02301/) (2015)
- [Event2Mind: Commonsense Inference on Events, Intents, and Reactions](https://mlsanity.com/paper/1805.06939/) (2018)
- [Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension](https://mlsanity.com/paper/2203.13947/) (2022)
- [SummScreen: A Dataset for Abstractive Screenplay Summarization](https://mlsanity.com/paper/2104.07091/) (2021)
- [A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task](https://mlsanity.com/paper/1606.02858/) (2016)
- [TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions](https://mlsanity.com/paper/2005.00242/) (2020)
- [An Annotated Dataset of Coreference in English Literature](https://mlsanity.com/paper/1912.01140/) (2019)
- ["Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative Understanding](https://mlsanity.com/paper/2109.05438/) (2021)
- [Movie Plot Analysis via Turning Point Identification](https://mlsanity.com/paper/1908.10328/) (2019)
- [To Test Machine Comprehension, Start by Defining Comprehension](https://mlsanity.com/paper/2005.01525/) (2020)
- [MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge](https://mlsanity.com/paper/1803.05223/) (2018)
- [ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning](https://mlsanity.com/paper/2104.08350/) (2021)
- [Exploring Content Selection in Summarization of Novel Chapters](https://mlsanity.com/paper/2005.01840/) (2020)
- [TellMeWhy: A Dataset for Answering Why-Questions in Narratives](https://mlsanity.com/paper/2106.06132/) (2021)
- [English Machine Reading Comprehension Datasets: A Survey](https://mlsanity.com/paper/2101.10421/) (2021)
- [Screenplay Summarization Using Latent Narrative Structure](https://mlsanity.com/paper/2004.12727/) (2020)
- [TVShowGuess: Character Comprehension in Stories as Speaker Guessing](https://mlsanity.com/paper/2204.07721/) (2022)
- [Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study](https://mlsanity.com/paper/2106.03826/) (2021)
- [Embracing data abundance: BookTest Dataset for Reading Comprehension](https://mlsanity.com/paper/1610.00956/) (2016)
- [Exploring aspects of similarity between spoken personal narratives by disentangling them into narrative clause types](https://mlsanity.com/paper/2005.12762/) (2020)
- [Annotating Character Relationships in Literary Texts](https://mlsanity.com/paper/1512.00728/) (2015)
- [CompRes: A Dataset for Narrative Structure in News](https://mlsanity.com/paper/2007.04874/) (2020)

## Built on by (most cited first, 10 of 10)

- [Training language models to summarize narratives improves brain alignment](https://mlsanity.com/paper/2212.10898/) (2022)
- [Personality Understanding of Fictional Characters during Book Reading](https://mlsanity.com/paper/2305.10156/) (2023)
- [Multi-level Contrastive Learning for Script-based Character Understanding](https://mlsanity.com/paper/2310.13231/) (2023)
- [MBTI Personality Prediction for Fictional Characters Using Movie Scripts](https://mlsanity.com/paper/2210.10994/) (2022)
- [Beyond Math: Stories as a Testbed for Memorization-Constrained Reasoning in LLMs](https://mlsanity.com/paper/2412.14368/) (2024)
- [Plot Retrieval as an Assessment of Abstract Semantic Association](https://mlsanity.com/paper/2311.01666/) (2023)
- [Previously on the Stories: Recap Snippet Identification for Story Reading](https://mlsanity.com/paper/2402.07271/) (2024)
- [Transferring Procedural Knowledge across Commonsense Tasks](https://mlsanity.com/paper/2304.13867/) (2023)
- [Characterizing the Effects of Translation on Intertextuality using Multilingual Embedding Spaces](https://mlsanity.com/paper/2501.10731/) (2025)
- [Whats New? Identifying the Unfolding of New Events in Narratives](https://mlsanity.com/paper/2302.07748/) (2023)

## Abstract (arXiv)

As the body of research on machine narrative comprehension grows, there is a critical need for consideration of performance assessment strategies as well as the depth and scope of different benchmark tasks. Based on narrative theories, reading comprehension theories, as well as existing machine narrative reading comprehension tasks and datasets, we propose a typology that captures the main similarities and differences among assessment tasks; and discuss the implications of our typology for new task design and the challenges of narrative reading comprehension.