mlsanityData to 14 Oct 2026

VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory

arXiv 2506.18903 ↗First posted 23 Jun 2025cs.CVcited by 109

Not on the map. Why?

Cited by 56 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a novel memory module for building video generators capable of interactively exploring environments. Previous approaches have achieved similar results either by out-painting 2D views of a scene while incrementally reconstructing its 3D geometry-which quickly accumulates errors-or by using video generators with a short context window, which struggle to maintain scene coherence over the long term. To address these limitations, we introduce Surfel-Indexed View Memory (VMem), a memory module that remembers past views by indexing them geometrically based on the 3D surface elements (surfels) they have observed. VMem enables efficient retrieval of the most relevant past views when generating new ones. By focusing only on these relevant views, our method produces consistent explorations of imagined environments at a fraction of the computational cost required to use all past views as context. We evaluate our approach on challenging long-term scene synthesis benchmarks and demonstrate superior performance compared to existing methods in maintaining scene coherence and camera control.

Key facts. Cited by 109 papers in the map (data through 14 Oct 2026). Builds on 39 of its 50 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

39 of 50 references.

References · 39

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  4. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  5. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  6. 2024Genie: Generative Interactive Environmentscited by 747
  7. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  8. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
  9. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  10. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  11. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  12. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
27 more
  1. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  2. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  3. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  4. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  5. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 364
  6. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  7. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 318
  8. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  9. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  10. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 243
  11. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
  12. 2022SPARF: Neural Radiance Fields from Sparse and Noisy Posescited by 195
  13. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 182
  14. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
  15. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  16. 2023WonderJourney: Going from Anywhere to Everywherecited by 158
  17. 2023Consistent View Synthesis with Pose-Guided Diffusion Modelscited by 117
  18. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 116
  19. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 116
  20. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 100
  21. 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
  22. 2023Long-Term Photometric Consistent Novel View Synthesis with Diffusion Modelscited by 76
  23. 2024GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warpingcited by 73
  24. 2024MultiDiff: Consistent Novel View Synthesis from a Single Imagecited by 68
  25. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  26. 2022Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Imagecited by 55
  27. 2025StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generationcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 18; 2026: 91.

109 papers.

Sort:

By area:

  1. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
  2. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 100
  3. 2025Mixture of Contexts for Long Video Generationcited by 95
  4. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 52
  5. 2025World-in-World: World Models in a Closed-Loop Worldcited by 47
  6. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 45
  7. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  8. 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worldscited by 44
  9. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 42
  10. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  11. 2025StoryMem: Multi-shot Long Video Storytelling with Memorycited by 40
  12. 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memorycited by 40
38 more
  1. 2026MIND: Benchmarking Memory Consistency and Action Control in World Modelscited by 35
  2. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 28
  3. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 28
  4. 2026MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokenscited by 27
  5. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Modelscited by 27
  6. 2026Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Modelscited by 26
  7. 2026VGGRPO: Towards World-Consistent Video Generation with 4D Latent Rewardcited by 24
  8. 2025EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memorycited by 20
  9. 2025Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generationcited by 20
  10. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  11. 2026Latent Spatial Memory for Video World Modelscited by 17
  12. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 16
  13. 2026LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generationcited by 16
  14. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
  15. 2026WorldKV: Efficient World Memory with World Retrieval and Compressioncited by 15
  16. 2026LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editingcited by 15
  17. 2026Addressable Memory for Video World Modelscited by 15
  18. 2026Code World Model: Coding Agent as World Braincited by 14
  19. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Modelscited by 13
  20. 2026Geometry-Aware Implicit Memory for Video World Modelscited by 13
  21. 2026Mode Seeking meets Mean Seeking for Fast Long Video Generationcited by 12
  22. 2026Grounding World Simulation Models in a Real-World Metropoliscited by 12
  23. 2026WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representationcited by 12
  24. 2026One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single Imagecited by 11
  25. 2026World-R1: Reinforcing 3D Constraints for Text-to-Video Generationcited by 11
  26. 2026ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modelingcited by 10
  27. 2026CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narrativescited by 10
  28. 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 10
  29. 2026Plenoptic Video Generationcited by 9
  30. 2026UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Modelscited by 9
  31. 2026MemLearner: Learning to Query Context memory for Video World Modelscited by 9
  32. 2026Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memorycited by 9
  33. 2025EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editingcited by 8
  34. 2026Geometry-as-context: Modulating Explicit 3D in Scene-consistent Video Generation to Geometry Contextcited by 8
  35. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 8
  36. 2026Mem-World: Memory-Augmented Action-Conditioned World Models for Persistent Robot Manipulationcited by 8
  37. 2025Learning Plug-and-play Memory for Guiding Video Diffusion Modelscited by 7
  38. 2026360Anything: Geometry-Free Lifting of Images and Videos to 360degcited by 7

50 most cited of 109 All 109 (JSON)