mlsanityData to 14 Oct 2026

Video World Models with Long-term Spatial Memory

arXiv 2506.05284 ↗First posted 5 Jun 2025cs.CVcited by 136

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Emerging world models autoregressively generate video frames in response to actions, such as camera movements and text prompts, among other control signals. Due to limited temporal context window sizes, these models often struggle to maintain scene consistency during revisits, leading to severe forgetting of previously generated environments. Inspired by the mechanisms of human memory, we introduce a novel framework to enhancing long-term consistency of video world models through a geometry-grounded long-term spatial memory. Our framework includes mechanisms to store and retrieve information from the long-term spatial memory and we curate custom datasets to train and evaluate world models with explicitly stored 3D memory mechanisms. Our evaluations show improved quality, consistency, and context length compared to relevant baselines, paving the way towards long-term consistent world generation.

Key facts. Cited by 136 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 79 of its 91 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

79 of 91 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  2. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
  3. 2024Open-Sora: Democratizing Efficient Video Production for All29 Dec
  4. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  5. 2024WorldSimBench: Towards Video Generation Models as World Simulators23 Oct
  6. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  7. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  8. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond6 May
  9. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  11. 2022Flow Matching for Generative Modeling6 Oct
  12. 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr
2 more
  1. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 65

  1. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2022Video Diffusion Modelscited by 2,305
  6. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  8. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  9. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  10. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  11. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  12. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
38 more
  1. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  2. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  3. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  4. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  5. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  6. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  7. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  8. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  9. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  10. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
  11. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  12. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  13. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 388
  14. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  15. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  16. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  17. 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 336
  18. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
  19. 2015Deep multi-scale video prediction beyond mean square errorcited by 300
  20. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  21. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  22. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  23. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 233
  24. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 229
  25. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  26. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 185
  27. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 180
  28. 2018Stochastic Adversarial Video Predictioncited by 173
  29. 2025History-Guided Video Diffusioncited by 171
  30. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  31. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 154
  32. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 146
  33. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 143
  34. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  35. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 125
  36. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 123
  37. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  38. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 98

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 37; 2026: 99.

136 papers.

Sort:

By area:

  1. 2025Controllable Video Generation: A Surveycited by 98
  2. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
  3. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 96
  4. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 94
  5. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 76
  6. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  7. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  8. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 42
  9. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  10. 2026MIND: Benchmarking Memory Consistency and Action Control in World Modelscited by 35
  11. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 28
  12. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Modelscited by 27
38 more
  1. 2026VideoMemory: Toward Consistent Video Generation via Memory Integrationcited by 25
  2. 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Modelscited by 23
  3. 2025Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generationcited by 21
  4. 2025EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memorycited by 20
  5. 2025Dexterous World Modelscited by 18
  6. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  7. 2026Latent Spatial Memory for Video World Modelscited by 17
  8. 2025BAgger: Backwards Aggregation for Mitigating Drift in Autoregressive Video Diffusion Modelscited by 16
  9. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 16
  10. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  11. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
  12. 2026Addressable Memory for Video World Modelscited by 15
  13. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generationcited by 14
  14. 2025Reinforcement Learning with Inverse Rewards for World Model Post-trainingcited by 13
  15. 20253D Scene Prompting for Scene-Consistent Camera-Controllable Video Generationcited by 13
  16. 2026ReRoPE: Repurposing RoPE for Relative Camera Controlcited by 13
  17. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Modelscited by 13
  18. 2026Geometry-Aware Implicit Memory for Video World Modelscited by 13
  19. 2026Geometry-Aware Rotary Position Embedding for Consistent Video World Modelcited by 12
  20. 2026Grounding World Simulation Models in a Real-World Metropoliscited by 12
  21. 2025How Much 3D Do Video Foundation Models Encode?cited by 11
  22. 2025SpaceTimePilot: Generative Rendering of Dynamic Scenes Across Space and Timecited by 11
  23. 2026World-R1: Reinforcing 3D Constraints for Text-to-Video Generationcited by 11
  24. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 11
  25. 2026LaVR: Scene Latent Conditioned Generative Video Trajectory Re-Rendering using Large 4D Reconstruction Modelscited by 10
  26. 2026ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modelingcited by 10
  27. 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 10
  28. 2026ReWorld: An Interactive World Model with Long-Horizon Memorycited by 10
  29. 2026Plenoptic Video Generationcited by 9
  30. 2026UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Modelscited by 9
  31. 2026MemLearner: Learning to Query Context memory for Video World Modelscited by 9
  32. 2025WorldPack: Dynamic Frame Compression for Long-context Video World Modelingcited by 8
  33. 2026Flow Equivariant World Models: Memory for Partially Observed Dynamic Environmentscited by 8
  34. 2026What-If World: A Causal Benchmark for General World Models in Embodied Scenarioscited by 8
  35. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 8
  36. 2025The World is Your Canvas: Painting Promptable Events with Reference Images, Trajectories, and Textcited by 7
  37. 2026360Anything: Geometry-Free Lifting of Images and Videos to 360degcited by 7
  38. 2026DreamWorld: Unified World Modeling in Video Generationcited by 7

50 most cited of 136 All 136 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Pixel Histories: World Models with Persistent 3D State
  2. 2026WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
  3. 2026World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models
  4. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Models
  5. 2025Long-Context State-Space Video World Models
  6. 2025WorldMem: Long-term Consistent World Simulation with Memory
  7. 2026LOCI: Spatial Linear Memory for Streaming World Models
  8. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling