# Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives

arXiv: https://arxiv.org/abs/1905.10847 · HTML: https://mlsanity.com/paper/1905.10847/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2019.

## Built on by (most cited first, 30 of 59)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Curriculum Temperature for Knowledge Distillation](https://mlsanity.com/paper/2211.16231/) (2022)
- [Efficient Methods for Natural Language Processing: A Survey](https://mlsanity.com/paper/2209.00099/) (2022)
- [Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences](https://mlsanity.com/paper/2403.07230/) (2024)
- [Principled Data Selection for Alignment: The Hidden Risks of Difficult Examples](https://mlsanity.com/paper/2502.09650/) (2025)
- [The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models](https://mlsanity.com/paper/2108.06084/) (2021)
- [Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Tracking](https://mlsanity.com/paper/2106.00291/) (2021)
- [CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Grounding](https://mlsanity.com/paper/2305.08685/) (2023)
- [Bridging Pre-trained Models and Downstream Tasks for Source Code Understanding](https://mlsanity.com/paper/2112.02268/) (2021)
- [Strategic Data Ordering: Enhancing Large Language Model Performance through Curriculum Learning](https://mlsanity.com/paper/2405.07490/) (2024)
- [ASP: Learn a Universal Neural Solver!](https://mlsanity.com/paper/2303.00466/) (2023)
- [A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generation](https://mlsanity.com/paper/2204.09867/) (2022)
- [CLNode: Curriculum Learning for Node Classification](https://mlsanity.com/paper/2206.07258/) (2022)
- [An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models](https://mlsanity.com/paper/2203.09486/) (2022)
- [Rethinking Search: Making Domain Experts out of Dilettantes](https://mlsanity.com/paper/2105.02274/) (2021)
- [Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study](https://mlsanity.com/paper/2106.03826/) (2021)
- [Curriculum Graph Machine Learning: A Survey](https://mlsanity.com/paper/2302.02926/) (2023)
- [DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing](https://mlsanity.com/paper/2212.03597/) (2022)
- [Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking](https://mlsanity.com/paper/2212.07617/) (2022)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [MM-CamObj: A Comprehensive Multimodal Dataset for Camouflaged Object Scenarios](https://mlsanity.com/paper/2409.16084/) (2024)
- [Learning Rate Curriculum](https://mlsanity.com/paper/2205.09180/) (2022)
- [When Do Curricula Work in Federated Learning?](https://mlsanity.com/paper/2212.12712/) (2022)
- [Reverse Stable Diffusion: What prompt was used to generate this image?](https://mlsanity.com/paper/2308.01472/) (2023)
- [HiCu: Leveraging Hierarchy for Curriculum Learning in Automated ICD Coding](https://mlsanity.com/paper/2208.02301/) (2022)
- [LCDnet: A Lightweight Crowd Density Estimation Model for Real-time Video Surveillance](https://mlsanity.com/paper/2302.05374/) (2023)
- [Digital Twin-based Anomaly Detection with Curriculum Learning in Cyber-physical Systems](https://mlsanity.com/paper/2309.15995/) (2023)
- [Investigating the Zone of Proximal Development of Language Models for In-Context Learning](https://mlsanity.com/paper/2502.06990/) (2025)

## Abstract (arXiv)

This paper tackles the problem of reading comprehension over long narratives where documents easily span over thousands of tokens. We propose a curriculum learning (CL) based Pointer-Generator framework for reading/sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty. This can be interpreted as a form of domain randomization and/or generative pretraining during training. To this end, the usage of the Pointer-Generator softens the requirement of having the answer within the context, enabling us to construct diverse training samples for learning. Additionally, we propose a new Introspective Alignment Layer (IAL), which reasons over decomposed alignments using block-based self-attention. We evaluate our proposed method on the NarrativeQA reading comprehension benchmark, achieving state-of-the-art performance, improving existing baselines by 51% relative improvement on BLEU-4 and 17% relative improvement on Rouge-L. Extensive ablations confirm the effectiveness of our proposed IAL and CL components.