mlsanityData to 7 Oct 2026

LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models

arXiv 2603.07145 ↗First posted 7 Mar 2026cs.CVcited by 20

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control. However, they implicitly assume that the world only evolves within the observer's field of view. Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime. In this work, we identify and formalize this overlooked limitation as the "out-of-sight dynamics" problem, which impedes video world models from representing a continuously evolving world. To address this issue, we propose LiveWorld, a novel framework that extends video world models to support persistent world evolution. Instead of treating the world as static observational memory, LiveWorld models a persistent global state composed of a static 3D background and dynamic entities that continue evolving even when unobserved. To maintain these unseen dynamics, LiveWorld introduces a monitor-based mechanism that autonomously simulates the temporal progression of active entities and synchronizes their evolved states upon revisiting, ensuring spatially coherent rendering. For evaluation, we further introduce LiveBench, a dedicated benchmark for the task of maintaining out-of-sight dynamics. Extensive experiments show that LiveWorld enables persistent event evolution and long-term scene consistency, bridging the gap between existing 2D observation-based memory and true 4D dynamic world simulation. The baseline and benchmark will be publicly available at https://zichengduan.github.io/LiveWorld/index.html.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 57 of its 69 references that are in the map. First posted on arXiv 7 Mar 2026.

Builds on

57 of 69 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026MIND: Benchmarking Memory Consistency and Action Control in World Models8 Feb
  2. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  3. 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
  4. 2025BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generation28 Nov
  5. 2025Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulation25 Nov
  6. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
  7. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  8. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  9. 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
  10. 2024WorldSimBench: Towards Video Generation Models as World Simulators23 Oct
  11. 2024Genie: Generative Interactive Environments23 Feb
  12. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov

Other references · 45

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  7. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  8. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  10. 2025SAM 3: Segment Anything with Conceptscited by 835
  11. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  12. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
33 more
  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  2. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  3. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  4. 2025VACE: All-in-One Video Creation and Editingcited by 467
  5. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  6. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  7. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
  8. 2024Navigation World Modelscited by 262
  9. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  10. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
  11. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  12. 2025History-Guided Video Diffusioncited by 165
  13. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
  14. 2024In-Context LoRA for Diffusion Transformerscited by 155
  15. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  16. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  17. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
  18. 2024CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Modelscited by 99
  19. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 97
  20. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
  21. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  22. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  23. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
  24. 2025Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compressioncited by 72
  25. 2025PAN: A World Model for General, Actionable, and Long-Horizon World Simulationcited by 36
  26. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 33
  27. 2025Whole-Body Conditioned Egocentric Video Predictioncited by 30
  28. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 20
  29. 2025Yan: Foundational Interactive Video Generationcited by 15
  30. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
  31. 2026Code2Worlds: Empowering Coding LLMs for 4D World Generationcited by 12
  32. 2026Flow Equivariant World Models: Memory for Partially Observed Dynamic Environmentscited by 6
  33. 2026GeoWorld: Geometric World Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 20.

20 papers.

Sort:

By area:

  1. 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generationcited by 11
  2. 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Modelscited by 10
  3. 2026Current World Models Lack a Persistent State Corecited by 8
  4. 2026Geometry-Aware Implicit Memory for Video World Modelscited by 7
  5. 2026Latent Spatial Memory for Video World Modelscited by 7
  6. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
  7. 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 4
  8. 2026WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memorycited by 4
  9. 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoningcited by 3
  10. 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memorycited by 2
  11. 2026UniMesh: Unifying 3D Mesh Understanding and Generationcited by 1
  12. 2026FlashSign: Pose-Free Guidance for Efficient Sign Language Video Generationcited by 0
8 more
  1. 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
  2. 2026Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioningcited by 0
  3. 2026Can 4D Foundation Models Remember?cited by 0
  4. 2026Code Plans, Diffusion Renders: Open-Ended Generative World Modelingcited by 0
  5. 2026DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Timecited by 0
  6. 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Modelscited by 0
  7. 20264Director: Controlling Video World Models with Rigid 3D Geometrycited by 0
  8. 2026World Observer: Joint Actor-Observer Generation for Persistent World Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Pixel Histories: World Models with Persistent 3D State
  2. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Models
  3. 2026World in World: Explore the World with World Models
  4. 2026MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
  5. 2026AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video
  6. 2025Learning 3D Persistent Embodied World Models
  7. 2026LIVE: Long-horizon Interactive Video World Modeling
  8. 2026Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistency