mlsanityData to 14 Oct 2026

Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval

arXiv 2506.03141 ↗First posted 3 Jun 2025cs.CVcited by 163

Not on the map. Why?

Cited by 79 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advances in interactive video generation have shown promising results, yet existing approaches struggle with scene-consistent memory capabilities in long video generation due to limited use of historical context. In this work, we propose Context-as-Memory, which utilizes historical context as memory for video generation. It includes two simple yet effective designs: (1) storing context in frame format without additional post-processing; (2) conditioning by concatenating context and frames to be predicted along the frame dimension at the input, requiring no external control modules. Furthermore, considering the enormous computational overhead of incorporating all historical context, we propose the Memory Retrieval module to select truly relevant context frames by determining FOV (Field of View) overlap between camera poses, which significantly reduces the number of candidate frames without substantial information loss. Experiments demonstrate that Context-as-Memory achieves superior memory capabilities in interactive long video generation compared to SOTAs, even generalizing effectively to open-domain scenarios not seen during training. The link of our project page is https://context-as-memory.github.io/.

Key facts. Cited by 163 papers in the map (data through 14 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

46 of 59 references.

References · 46

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  10. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  12. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
34 more
  1. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  2. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  3. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  4. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  5. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  6. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  7. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  8. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  9. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  10. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  11. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  12. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  13. 2023Learning Interactive Real-World Simulatorscited by 458
  14. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  15. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  16. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  17. 2024Diffusion Models Are Real-Time Game Enginescited by 296
  18. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  19. 2024Autoregressive Video Generation without Vector Quantizationcited by 186
  20. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 185
  21. 2025History-Guided Video Diffusioncited by 171
  22. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 168
  23. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 160
  24. 2023WonderJourney: Going from Anywhere to Everywherecited by 158
  25. 2025GameFactory: Creating New Games with Generative Interactive Videoscited by 130
  26. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 123
  27. 2025Long Context Tuning for Video Generationcited by 113
  28. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 93
  29. 2024Video as the New Language for Real-World Decision Makingcited by 87
  30. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 80
  31. 20243DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generationcited by 71
  32. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  33. 2025A Survey of Interactive Generative Videocited by 23
  34. 2025Position: Interactive Generative Video as Next-Generation Game Enginecited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 37; 2026: 126.

163 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
  3. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 100
  4. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 96
  5. 2025Mixture of Contexts for Long Video Generationcited by 95
  6. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 77
  7. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 62
  8. 2026Context Forcing: Consistent Autoregressive Video Generation with Long Contextcited by 59
  9. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 52
  10. 2025World-in-World: World Models in a Closed-Loop Worldcited by 47
  11. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 45
  12. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
38 more
  1. 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worldscited by 44
  2. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 42
  3. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  4. 2025StoryMem: Multi-shot Long Video Storytelling with Memorycited by 40
  5. 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memorycited by 40
  6. 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 40
  7. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 38
  8. 2025MultiShotMaster: A Controllable Multi-Shot Video Generation Frameworkcited by 36
  9. 2025OneStory: Coherent Multi-Shot Video Generation with Adaptive Memorycited by 35
  10. 2026MIND: Benchmarking Memory Consistency and Action Control in World Modelscited by 35
  11. 2026Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstructioncited by 31
  12. 2025Pack and Force Your Memory: Long-form and Consistent Video Generationcited by 29
  13. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 28
  14. 2026MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokenscited by 27
  15. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Modelscited by 27
  16. 2026Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Modelscited by 26
  17. 2026VideoMemory: Toward Consistent Video Generation via Memory Integrationcited by 25
  18. 2026Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generationcited by 24
  19. 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Modelscited by 23
  20. 2025Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesiscited by 21
  21. 2025Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generationcited by 20
  22. 2025VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement Learningcited by 18
  23. 2025MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generationcited by 18
  24. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  25. 2026Latent Spatial Memory for Video World Modelscited by 17
  26. 2026LIVE: Long-horizon Interactive Video World Modelingcited by 16
  27. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 16
  28. 2026Anchor Forcing: Anchor Memory and Tri-Region RoPE for Interactive Streaming Video Diffusioncited by 16
  29. 2026OmniRoam: World Wandering via Long-Horizon Panoramic Video Generationcited by 16
  30. 2026LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generationcited by 16
  31. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  32. 2026Pathwise Test-Time Correction for Autoregressive Long Video Generationcited by 15
  33. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
  34. 2026Gamma-World: Generative Multi-Agent World Modeling Beyond Two Playerscited by 15
  35. 2026Wonder: Video World Model Done Bettercited by 15
  36. 2026Addressable Memory for Video World Modelscited by 15
  37. 2025Mitty: Diffusion-based Human-to-Robot Video Generationcited by 14
  38. 2026Code World Model: Coding Agent as World Braincited by 14

50 most cited of 163 All 163 (JSON)