mlsanityData to 7 Oct 2026

The NarrativeQA Reading Comprehension Challenge

arXiv 1712.07040 ↗First posted 19 Dec 2017cs.CL cs.AI cs.NEcited by 590

Not on the map. Why?

Cited by 77 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reading comprehension (RC)---in contrast to information retrieval---requires integrating information and reasoning about events, entities, and their relations across a full document. Question answering is conventionally used to assess RC ability, in both artificial agents and children learning to read. However, existing RC datasets and tasks are dominated by questions that can be solved by selecting answers using superficial information (e.g., local context similarity or global term frequency); they thus fail to test for the essential integrative aspect of RC. To encourage progress on deeper comprehension of language, we present a new dataset and set of tasks in which the reader must answer questions about stories by reading entire books or movie scripts. These tasks are designed so that successfully answering their questions requires understanding the underlying narrative rather than relying on shallow pattern matching or salience. We show that although humans solve the tasks easily, standard RC models struggle on the tasks presented here. We provide an analysis of the dataset and the challenges it presents.

Key facts. Cited by 590 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 52; 2022: 48; 2023: 79; 2024: 141; 2025: 151; 2026: 119.

590 papers.

Sort:

By area:

  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  5. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  6. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  7. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
  8. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  9. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  10. 2021MetaICL: Learning to Learn In Contextcited by 450
  11. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  12. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 390
38 more
  1. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  2. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 360
  3. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
  4. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 295
  5. 2022UL2: Unifying Language Learning Paradigmscited by 292
  6. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  7. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  8. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  9. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  10. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  11. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 242
  12. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  13. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 235
  14. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 232
  15. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  16. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  17. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 204
  18. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  19. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
  20. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  21. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 199
  22. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  23. 2025Survey on Evaluation of LLM-based Agentscited by 190
  24. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  25. 2024An Empirical Study of Mamba-based Language Modelscited by 177
  26. 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 175
  27. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  28. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  29. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  30. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  31. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 146
  32. 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 142
  33. 2024How to Train Long-Context Language Models (Effectively)cited by 141
  34. 2024A Human-Inspired Reading Agent with Gist Memory of Very Long Contextscited by 140
  35. 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 127
  36. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  37. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  38. 2022A Length-Extrapolatable Transformercited by 123

50 most cited of 590 All 590 (JSON)