NarrativeWorldBench: A Frontier-Saturated Benchmark and a Latent World Model for Long-Horizon Co-Creative Audio Drama
On the map: World models → World models for language agents
Abstract (arXiv)
Long-form serialized audio drama, with arcs that run for 200 to 800 episodes, is a major creative medium and a setting where frontier large language models (LLMs) fail. We benchmark 21 models, spanning classical, fine-tuned, open-frontier, closed-frontier, and reasoning tiers, on a uniform set of structural narrative metrics. All closed-frontier systems saturate at a plot-beat F1 in the band [0.78, 0.81] and collapse by about -0.20 F1 at horizon h=200. We introduce NarrativeWorldBench, an open benchmark of nine narrative-structure metrics evaluated across horizons h in {10, 20, 50, 100, 200}, with cross-lingual evaluation across four Indic languages (Hindi, Tamil, Telugu, Marathi). We introduce N-VSSM, a Narrative Variational State-Space Model that maintains a structured 256-dimensional latent world state over more than 200 episodes via a Mamba-2 backbone with an event-conditioned posterior and an 8B decoder. N-VSSM holds plot-beat F1 >= 0.84 across all horizons at 4x lower compute than the closed-frontier band. A learned Cultural Transfer Function lifts cross-language fidelity by +0.20 to +0.23 Likert points. In a within-subjects writer study (n = 12 professional authors, 240 trials), N-VSSM is preferred over Claude Opus 4.5 on long-arc consistency 71% of the time and rated +1.3 Likert points higher on controllability.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 47 of its 60 references that are in the map. First posted on arXiv 16 Jun 2026.
Builds on
47 of 60 references.
References · 47
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2013Auto-Encoding Variational Bayescited by 4,053
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
- 2018World Modelscited by 1,513
35 more
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2024Genie: Generative Interactive Environmentscited by 747
- 2015Generating Sentences from a Continuous Spacecited by 612
- 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
- 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
- 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 417
- 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
- 2021Recursively Summarizing Books with Human Feedbackcited by 290
- 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 249
- 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 234
- 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 233
- 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
- 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
- 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 183
- 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 167
- 2024Investigating Cultural Alignment of Large Language Modelscited by 143
- 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 112
- 2025WritingBench: A Comprehensive Benchmark for Generative Writingcited by 106
- 2023RecurrentGPT: Interactive Generation of (Arbitrarily) Long Textcited by 84
- 2024One Thousand and One Pairs: A "novel" challenge for long-context language modelscited by 80
- 2022Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writingcited by 71
- 2022Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systemscited by 59
- 2020STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generationcited by 56
- 2021Wordcraft: a Human-AI Collaborative Editor for Story Writingcited by 41
- 2024PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Datacited by 37
- 2019Movie Plot Analysis via Turning Point Identificationcited by 34
- 2025One ruler to measure them all: Benchmarking multilingual long-context language modelscited by 19
- 2024MoPS: Modular Story Premise Synthesis for Open-Ended Automatic Story Generationcited by 13
- 2025Fluent but Foreign: Even Regional LLMs Lack Cultural Alignmentcited by 10
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025SeriesBench: A Benchmark for Narrative-Driven Drama Series Understanding
- 2026Narrative World Model: Narratology-Grounded Writer Memory for Long-Form Fiction
- 2025SCORE: Story Coherence and Retrieval Enhancement for AI Narratives
- 2026Scaling Long-Form Story Generation via Narrative State Tracking
- 2026From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form Narratives
- 2025SeqBench: Benchmarking Sequential Narrative Generation in Text-to-Video Models
- 2026Beyond Sparse Rewards: A New Benchmark and Structure-Aware Graph Alignment for Micro-Drama Understanding
- 2025NexusSum: Hierarchical LLM Agents for Long-Form Narrative Summarization