mlsanityData to 7 Oct 2026

Long-Context State-Space Video World Models

arXiv 2505.20171 ↗First posted 26 May 2025cs.CVcited by 59

On the map: World models → Video-generation world models and simulators

also in 1 more
Abstract (arXiv)

Video diffusion models have recently shown promise for world modeling through autoregressive frame prediction conditioned on actions. However, they struggle to maintain long-term memory due to the high computational cost associated with processing extended sequences in attention layers. To overcome this limitation, we propose a novel architecture leveraging state-space models (SSMs) to extend temporal memory without compromising computational efficiency. Unlike previous approaches that retrofit SSMs for non-causal vision tasks, our method fully exploits the inherent advantages of SSMs in causal sequence modeling. Central to our design is a block-wise SSM scanning scheme, which strategically trades off spatial consistency for extended temporal memory, combined with dense local attention to ensure coherence between consecutive frames. We evaluate the long-term memory capabilities of our model through spatial retrieval and reasoning tasks over extended horizons. Experiments on Memory Maze and Minecraft datasets demonstrate that our approach surpasses baselines in preserving long-range memory, while maintaining practical inference speeds suitable for interactive applications.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 79 of its 89 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

79 of 89 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
  2. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  3. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  5. 2024Pandora: Towards General World Model with Natural Language Actions and Video States12 Jun
  6. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond6 May
  7. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens18 Jan
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  9. 2023Learning Interactive Real-World Simulators9 Oct
  10. 2022Scalable Diffusion Models with Transformers19 Dec
  11. 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
1 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2016Layer Normalizationcited by 5,579
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  8. 2022Video Diffusion Modelscited by 2,280
  9. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  10. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  12. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
38 more
  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  2. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  3. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  4. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  5. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  6. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  7. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  8. 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
  9. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  10. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
  11. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  12. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  13. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  14. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  15. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  16. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  17. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 373
  18. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  19. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
  20. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
  21. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  22. 2022Transformers are Sample-Efficient World Modelscited by 317
  23. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  24. 2024Titans: Learning to Memorize at Test Timecited by 303
  25. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  26. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
  27. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  28. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 180
  29. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  30. 2024Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingcited by 167
  31. 2025History-Guided Video Diffusioncited by 165
  32. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 163
  33. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
  34. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  35. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 79
  36. 2023Diffusion Models Without Attentioncited by 75
  37. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Modelscited by 74
  38. 2024DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesiscited by 72

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 23; 2026: 36.

59 papers.

Sort:

By area:

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  2. 2025Memory in the Age of AI Agentscited by 285
  3. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 174
  4. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  5. 2025Cameras as Relative Positional Encodingcited by 94
  6. 2025Mixture of Contexts for Long Video Generationcited by 90
  7. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 89
  8. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 86
  9. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  10. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  11. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 46
  12. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 45
38 more
  1. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 39
  2. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 27
  3. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 26
  4. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  5. 2026Anchor Forcing: Anchor Memory and Tri-Region RoPE for Interactive Streaming Video Diffusioncited by 14
  6. 2025BAgger: Backwards Aggregation for Mitigating Drift in Autoregressive Video Diffusion Modelscited by 13
  7. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
  8. 2026Mode Seeking meets Mean Seeking for Fast Long Video Generationcited by 12
  9. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Modelscited by 12
  10. 2026WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representationcited by 12
  11. 2026VideoWorld 2: Learning Transferable Knowledge from Real-world Videoscited by 11
  12. 2026Beyond Pixel Histories: World Models with Persistent 3D Statecited by 11
  13. 2026Addressable Memory for Video World Modelscited by 11
  14. 2025Recurrent Autoregressive Diffusion: Global Memory Meets Local Attentioncited by 9
  15. 2026MemLearner: Learning to Query Context memory for Video World Modelscited by 8
  16. 2025WorldPack: Dynamic Frame Compression for Long-context Video World Modelingcited by 7
  17. 2025The World is Your Canvas: Painting Promptable Events with Reference Images, Trajectories, and Textcited by 7
  18. 2025Forecasting When to Forecast: Accelerating Diffusion Models with Confidence-Gated Taylorcited by 6
  19. 2026Flow Equivariant World Models: Memory for Partially Observed Dynamic Environmentscited by 6
  20. 2026A Mechanistic View on Video Generation as World Models: State and Dynamicscited by 6
  21. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 6
  22. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
  23. 2025From Masks to Worlds: A Hitchhiker's Guide to World Modelscited by 5
  24. 2026DIM-WAM: World-Action Modeling with Diverse Historical Event Memorycited by 5
  25. 2026Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusioncited by 5
  26. 2025AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Pathcited by 4
  27. 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Modelscited by 4
  28. 2026Soap2Soap: Long Cinematic Video Remaking via Multi-Agent Collaborationcited by 4
  29. 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 4
  30. 2026AlayaWorld: Long-Horizon and Playable Video World Generationcited by 4
  31. 2026From Pixels to States: Rethinking Interactive World Models as Game Enginescited by 4
  32. 2026Attend Locally, Remember Linearly: Linear Attention as Cross-Frame Memory for Autoregressive Video Diffusioncited by 3
  33. 2026Composition of Memory Experts for Diffusion World Modelscited by 3
  34. 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applicationscited by 3
  35. 2025On Memory: A comparison of memory mechanisms in world modelscited by 2
  36. 2026Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editingcited by 2
  37. 2026Foveated Diffusion: Efficient Spatially Adaptive Image and Video Generationcited by 2
  38. 2026WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Modelscited by 2

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Video World Models with Long-term Spatial Memory
  2. 2026Spatial Memory Intelligence: Endowing World Models with Understanding-Driven Long-Term Memory
  3. 2025StateSpaceDiffuser: Bringing Long Context to Diffusion World Models
  4. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Models
  5. 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memory
  6. 2026LongSpace: Exploring Long-Horizon Spatial Memory from Perception to Recall in Video
  7. 2025Pushing the Boundaries of State Space Models for Image and Video Generation
  8. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Models