mlsanityData to 14 Oct 2026

NarrativeWorldBench: A Frontier-Saturated Benchmark and a Latent World Model for Long-Horizon Co-Creative Audio Drama

arXiv 2606.17391 ↗First posted 16 Jun 2026cs.CL cs.AI cs.LGcited by 0

On the map: World models → World models for language agents

Abstract (arXiv)

Long-form serialized audio drama, with arcs that run for 200 to 800 episodes, is a major creative medium and a setting where frontier large language models (LLMs) fail. We benchmark 21 models, spanning classical, fine-tuned, open-frontier, closed-frontier, and reasoning tiers, on a uniform set of structural narrative metrics. All closed-frontier systems saturate at a plot-beat F1 in the band [0.78, 0.81] and collapse by about -0.20 F1 at horizon h=200. We introduce NarrativeWorldBench, an open benchmark of nine narrative-structure metrics evaluated across horizons h in {10, 20, 50, 100, 200}, with cross-lingual evaluation across four Indic languages (Hindi, Tamil, Telugu, Marathi). We introduce N-VSSM, a Narrative Variational State-Space Model that maintains a structured 256-dimensional latent world state over more than 200 episodes via a Mamba-2 backbone with an event-conditioned posterior and an 8B decoder. N-VSSM holds plot-beat F1 >= 0.84 across all horizons at 4x lower compute than the closed-frontier band. A learned Cultural Transfer Function lifts cross-language fidelity by +0.20 to +0.23 Likert points. In a within-subjects writer study (n = 12 professional authors, 240 trials), N-VSSM is preferred over Claude Opus 4.5 on long-arc consistency 71% of the time and rated +1.3 Likert points higher on controllability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 47 of its 60 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

47 of 60 references.

References · 47

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  4. 2013Auto-Encoding Variational Bayescited by 4,053
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  6. 2018Neural Ordinary Differential Equationscited by 3,779
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  8. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  9. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  10. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  11. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  12. 2018World Modelscited by 1,513
35 more
  1. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  2. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  3. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  4. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  6. 2023Mastering Diverse Domains through World Modelscited by 1,253
  7. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  8. 2024Genie: Generative Interactive Environmentscited by 747
  9. 2015Generating Sentences from a Continuous Spacecited by 612
  10. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  11. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  12. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 417
  13. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
  14. 2021Recursively Summarizing Books with Human Feedbackcited by 290
  15. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 249
  16. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 234
  17. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 233
  18. 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
  19. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  20. 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 183
  21. 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 167
  22. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
  23. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 112
  24. 2025WritingBench: A Comprehensive Benchmark for Generative Writingcited by 106
  25. 2023RecurrentGPT: Interactive Generation of (Arbitrarily) Long Textcited by 84
  26. 2024One Thousand and One Pairs: A "novel" challenge for long-context language modelscited by 80
  27. 2022Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writingcited by 71
  28. 2022Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systemscited by 59
  29. 2020STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generationcited by 56
  30. 2021Wordcraft: a Human-AI Collaborative Editor for Story Writingcited by 41
  31. 2024PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Datacited by 37
  32. 2019Movie Plot Analysis via Turning Point Identificationcited by 34
  33. 2025One ruler to measure them all: Benchmarking multilingual long-context language modelscited by 19
  34. 2024MoPS: Modular Story Premise Synthesis for Open-Ended Automatic Story Generationcited by 13
  35. 2025Fluent but Foreign: Even Regional LLMs Lack Cultural Alignmentcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SeriesBench: A Benchmark for Narrative-Driven Drama Series Understanding
  2. 2026Narrative World Model: Narratology-Grounded Writer Memory for Long-Form Fiction
  3. 2025SCORE: Story Coherence and Retrieval Enhancement for AI Narratives
  4. 2026Scaling Long-Form Story Generation via Narrative State Tracking
  5. 2026From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form Narratives
  6. 2025SeqBench: Benchmarking Sequential Narrative Generation in Text-to-Video Models
  7. 2026Beyond Sparse Rewards: A New Benchmark and Structure-Aware Graph Alignment for Micro-Drama Understanding
  8. 2025NexusSum: Hierarchical LLM Agents for Long-Form Narrative Summarization