# Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study

arXiv: https://arxiv.org/abs/2106.03826 · HTML: https://mlsanity.com/paper/2106.03826/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine](https://mlsanity.com/paper/1704.05179/) (2017)
- [Show Your Work: Improved Reporting of Experimental Results](https://mlsanity.com/paper/1909.03004/) (2019)
- [Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model](https://mlsanity.com/paper/1912.09637/) (2019)
- [A Discrete Hard EM Approach for Weakly Supervised Question Answering](https://mlsanity.com/paper/1909.04849/) (2019)
- [Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation](https://mlsanity.com/paper/1706.09799/) (2017)
- [Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives](https://mlsanity.com/paper/1905.10847/) (2019)
- [TWEETQA: A Social Media Focused Question Answering Dataset](https://mlsanity.com/paper/1907.06292/) (2019)
- [Quasar: Datasets for Question Answering by Search and Reading](https://mlsanity.com/paper/1707.03904/) (2017)
- [Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering](https://mlsanity.com/paper/1711.05116/) (2017)
- [Improving Machine Reading Comprehension with General Reading Strategies](https://mlsanity.com/paper/1810.13441/) (2018)
- [Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering](https://mlsanity.com/paper/2005.01898/) (2020)
- [Multi-style Generative Reading Comprehension](https://mlsanity.com/paper/1901.02262/) (2019)
- [ReadTwice: Reading Very Large Documents with Memories](https://mlsanity.com/paper/2105.04241/) (2021)
- [RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering](https://mlsanity.com/paper/2010.10757/) (2020)
- [Frustratingly Hard Evidence Retrieval for QA Over Books](https://mlsanity.com/paper/2007.09878/) (2020)
- [Complementary Evidence Identification in Open-Domain Question Answering](https://mlsanity.com/paper/2103.11643/) (2021)

## Built on by (most cited first, 16 of 16)

- [Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension](https://mlsanity.com/paper/2203.13947/) (2022)
- [Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Tasks](https://mlsanity.com/paper/2304.14732/) (2023)
- [ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers](https://mlsanity.com/paper/2110.06884/) (2021)
- [It is AI's Turn to Ask Humans a Question: Question-Answer Pair Generation for Children's Story Books](https://mlsanity.com/paper/2109.03423/) (2021)
- [Personality Understanding of Fictional Characters during Book Reading](https://mlsanity.com/paper/2305.10156/) (2023)
- [TVShowGuess: Character Comprehension in Stories as Speaker Guessing](https://mlsanity.com/paper/2204.07721/) (2022)
- [Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations](https://mlsanity.com/paper/2305.03117/) (2023)
- [A Survey of Machine Narrative Reading Comprehension Assessments](https://mlsanity.com/paper/2205.00299/) (2022)
- [SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension](https://mlsanity.com/paper/2508.01959/) (2025)
- [MultiVENT: Multilingual Videos of Events with Aligned Natural Text](https://mlsanity.com/paper/2307.03153/) (2023)
- [LiteraryQA: Towards Effective Evaluation of Long-document Narrative QA](https://mlsanity.com/paper/2510.13494/) (2025)
- [TIGQA:An Expert Annotated Question Answering Dataset in Tigrinya](https://mlsanity.com/paper/2404.17194/) (2024)
- [Plot Retrieval as an Assessment of Abstract Semantic Association](https://mlsanity.com/paper/2311.01666/) (2023)
- [Equipping Transformer with Random-Access Reading for Long-Context Understanding](https://mlsanity.com/paper/2405.13216/) (2024)
- [DragonVerseQA: Open-Domain Long-Form Context-Aware Question-Answering](https://mlsanity.com/paper/2412.16694/) (2024)
- [GeopoliticalQA: Benchmarking Retrieval and Generation over Intelligence Reports](https://mlsanity.com/paper/2610.12617/) (2026)

## Abstract (arXiv)

Recent advancements in open-domain question answering (ODQA), i.e., finding answers from large open-domain corpus like Wikipedia, have led to human-level performance on many datasets. However, progress in QA over book stories (Book QA) lags behind despite its similar task formulation to ODQA. This work provides a comprehensive and quantitative analysis about the difficulty of Book QA: (1) We benchmark the research on the NarrativeQA dataset with extensive experiments with cutting-edge ODQA techniques. This quantifies the challenges Book QA poses, as well as advances the published state-of-the-art with a ~7% absolute improvement on Rouge-L. (2) We further analyze the detailed challenges in Book QA through human studies.footnoteurlhttps://github.com/gorov/BookQA. Our findings indicate that the event-centric questions dominate this task, which exemplifies the inability of existing QA models to handle event-oriented scenarios.