# StoryMem: Multi-shot Long Video Storytelling with Memory

arXiv: https://arxiv.org/abs/2512.19539 · HTML: https://mlsanity.com/paper/2512.19539/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 45 of its 53 references that are in the map.
- First posted on arXiv 22 Dec 2025.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation](https://mlsanity.com/paper/2307.06942/) (2023)
- [StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation](https://mlsanity.com/paper/2405.01434/) (2024)
- [HPSv3: Towards Wide-Spectrum Human Preference Score](https://mlsanity.com/paper/2508.03789/) (2025)
- [WorldMem: Long-term Consistent World Simulation with Memory](https://mlsanity.com/paper/2504.12369/) (2025)
- [Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval](https://mlsanity.com/paper/2506.03141/) (2025)
- [In-Context LoRA for Diffusion Transformers](https://mlsanity.com/paper/2410.23775/) (2024)
- [StoryGAN: A Sequential Conditional GAN for Story Visualization](https://mlsanity.com/paper/1812.02784/) (2018)
- [Training-Free Consistent Text-to-Image Generation](https://mlsanity.com/paper/2402.03286/) (2024)
- [From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs](https://mlsanity.com/paper/2504.15965/) (2025)
- [Long Context Tuning for Video Generation](https://mlsanity.com/paper/2503.10589/) (2025)
- [VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory](https://mlsanity.com/paper/2506.18903/) (2025)
- [Mixture of Contexts for Long Video Generation](https://mlsanity.com/paper/2508.21058/) (2025)
- [MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequence](https://mlsanity.com/paper/2407.16655/) (2024)
- [Automated Movie Generation via Multi-Agent CoT Planning](https://mlsanity.com/paper/2503.07314/) (2025)

## Built on by (most cited first, 30 of 40)

- [Mode Seeking meets Mean Seeking for Fast Long Video Generation](https://mlsanity.com/paper/2602.24289/) (2026)
- [ViMax: Agentic Video Generation](https://mlsanity.com/paper/2606.07649/) (2026)
- [ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation](https://mlsanity.com/paper/2603.11421/) (2026)
- [VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generation](https://mlsanity.com/paper/2606.08091/) (2026)
- [A2RD: Agentic Autoregressive Diffusion for Long Video Consistency](https://mlsanity.com/paper/2605.06924/) (2026)
- [EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generation](https://mlsanity.com/paper/2605.15199/) (2026)
- [One Sentence, One Drama: Personalized Short-Form Drama Generation via Multi-Agent Systems](https://mlsanity.com/paper/2605.22144/) (2026)
- [CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation](https://mlsanity.com/paper/2606.09639/) (2026)
- [Echo-Memory: A Controlled Study of Memory in Action World Models](https://mlsanity.com/paper/2606.09803/) (2026)
- [UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating](https://mlsanity.com/paper/2606.21661/) (2026)
- [Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms](https://mlsanity.com/paper/2603.28489/) (2026)
- [Soap2Soap: Long Cinematic Video Remaking via Multi-Agent Collaboration](https://mlsanity.com/paper/2605.17423/) (2026)
- [Memento: Reconstruct to Remember for Consistent Long Video Generation](https://mlsanity.com/paper/2606.14667/) (2026)
- [GroundShot: Visually Consistent Multi-Shot Long Video Generation via Entity-Grounded Shot Scheduling](https://mlsanity.com/paper/2606.20799/) (2026)
- [Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds](https://mlsanity.com/paper/2608.23383/) (2026)
- [StoryBlender: Inter-Shot Consistent and Editable 3D Storyboard with Spatial-temporal Dynamics](https://mlsanity.com/paper/2604.03315/) (2026)
- [CANVAS: Continuity-Aware Narratives via Visual Agentic Storyboarding](https://mlsanity.com/paper/2604.13452/) (2026)
- [SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation](https://mlsanity.com/paper/2605.09442/) (2026)
- [EM-Vid: Training-Free Entity-Centric Memory for Efficient and Consistent Multi-Shot Video Generation](https://mlsanity.com/paper/2605.23610/) (2026)
- [RECAP-Forcing: Retaining Content Appearances for Long Video Generation](https://mlsanity.com/paper/2608.26671/) (2026)
- [EvoState: Closed-Loop Visual State Management for Long-Form Video Generation](https://mlsanity.com/paper/2606.16184/) (2026)
- [FilmWorld: Agentic Novel-to-Film Generation through Dynamic Cinematic World Modeling](https://mlsanity.com/paper/2607.19038/) (2026)
- [Vorch-Director: Interactive World Story Model via Noise-Aware Error Rectification](https://mlsanity.com/paper/2608.05776/) (2026)
- [LogiShot: Logically Coherent Cross-Shot Video Generation](https://mlsanity.com/paper/2608.08820/) (2026)
- [SubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytelling](https://mlsanity.com/paper/2609.34502/) (2026)
- [HECTOR: Hybrid Editable Compositional Object References for Video Generation](https://mlsanity.com/paper/2603.08850/) (2026)
- [GO-Renderer: Generative Object Rendering with 3D-aware Controllable Video Diffusion Models](https://mlsanity.com/paper/2603.23246/) (2026)
- [Long-CODE: Isolating Pure Long-Context as an Orthogonal Dimension in Video Evaluation](https://mlsanity.com/paper/2604.17428/) (2026)
- [MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control](https://mlsanity.com/paper/2606.29473/) (2026)
- [KathaTrace: Diagnosing Semantic Trajectory Collapse in Generated Visual Narratives](https://mlsanity.com/paper/2607.01312/) (2026)

## Abstract (arXiv)

Visual storytelling requires generating multi-shot videos with cinematic quality and long-range consistency. Inspired by human memory, we propose StoryMem, a paradigm that reformulates long-form video storytelling as iterative shot synthesis conditioned on explicit visual memory, transforming pre-trained single-shot video diffusion models into multi-shot storytellers. This is achieved by a novel Memory-to-Video (M2V) design, which maintains a compact and dynamically updated memory bank of keyframes from historical generated shots. The stored memory is then injected into single-shot video diffusion models via latent concatenation and negative RoPE shifts with only LoRA fine-tuning. A semantic keyframe selection strategy, together with aesthetic preference filtering, further ensures informative and stable memory throughout generation. Moreover, the proposed framework naturally accommodates smooth shot transitions and customized story generation applications. To facilitate evaluation, we introduce ST-Bench, a diverse benchmark for multi-shot video storytelling. Extensive experiments demonstrate that StoryMem achieves superior cross-shot consistency over previous methods while preserving high aesthetic quality and prompt adherence, marking a significant step toward coherent minute-long video storytelling.