STAGE: A Full-Screenplay Benchmark for Reasoning over Evolving Stories
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Movie screenplays are a demanding testbed for long-form narrative understanding, as characters' goals, beliefs, knowledge, and relationships evolve continuously across scenes. However, existing benchmarks primarily evaluate isolated facts from the completed screenplay, leaving unassessed whether models can track the evolving state of characters as the story unfolds. We introduce STAGE, a benchmark over 151 English and Chinese full-length screenplays, built on a provenance-linked narrative backbone that recovers the state and epistemic access of each character at every point along its timeline. Three tasks derived from the backbone jointly probe whether models can maintain, explain, and act on evolving narrative state: Character Development Tracking updates a focal character's state between checkpoints, Cross-Scene Narrative Evolution Reasoning targets cross-scene state transitions, and In-Script Character Role-Playing requires responses bounded by the character's state and knowledge at a specified point. We identify three failure modes of current LLMs: silent forgetting under recursive state updating, limited cross-scene reasoning even when all relevant evidence is supplied, and a trade-off in role-playing where stylistic character fidelity and screenplay-grounded memory faithfulness are optimized by different memory-access strategies. STAGE thus provides a unified framework for diagnosing how current models fail to track, reason about, and enact story evolution.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 10 of its 65 references that are in the map. First posted on arXiv 13 Jan 2026.
Builds on
10 of 65 references.
Closest ancestors · 1
Same direction, newest first
- 2025Qwen3 Technical Report14 May
Other references · 9
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
- 2024NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokenscited by 32
- 2025Consistency in Language Models: Current Landscape, Challenges, and Future Directionscited by 14
- 2025Too Long, Didn't Model: Decomposing LLM Long-Context Understanding With Novelscited by 9
- 2025CML-Bench: A Framework for Evaluating and Enhancing LLM-Powered Movie Scripts Generationcited by 7
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Narrative World Model: Narratology-Grounded Writer Memory for Long-Form Fiction
- 2026EvolvingWorld: An Open-Schema Framework for Co-Evolving Role-Play Agents and World Model in Interactive Literary World
- 2026Can LLM Agents Stick to the Script? A Benchmark for Long-Horizon Consistency in Interactive Narratives
- 2026Staying In Character: Perspective-Bounded Memory For Book-Based Role-Playing Agents
- 2026Planning Beyond Text: Graph-based Reasoning for Complex Narrative Generation
- 2026Shared Worlds, Private Minds: Structured Memory for Long-Form Writing as World Creation
- 2025PRELUDE: A Benchmark Designed to Require Global Comprehension and Reasoning over Long Contexts
- 2026NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding