LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control. However, they implicitly assume that the world only evolves within the observer's field of view. Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime. In this work, we identify and formalize this overlooked limitation as the "out-of-sight dynamics" problem, which impedes video world models from representing a continuously evolving world. To address this issue, we propose LiveWorld, a novel framework that extends video world models to support persistent world evolution. Instead of treating the world as static observational memory, LiveWorld models a persistent global state composed of a static 3D background and dynamic entities that continue evolving even when unobserved. To maintain these unseen dynamics, LiveWorld introduces a monitor-based mechanism that autonomously simulates the temporal progression of active entities and synchronizes their evolved states upon revisiting, ensuring spatially coherent rendering. For evaluation, we further introduce LiveBench, a dedicated benchmark for the task of maintaining out-of-sight dynamics. Extensive experiments show that LiveWorld enables persistent event evolution and long-term scene consistency, bridging the gap between existing 2D observation-based memory and true 4D dynamic world simulation. The baseline and benchmark will be publicly available at https://zichengduan.github.io/LiveWorld/index.html.
Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 57 of its 69 references that are in the map. First posted on arXiv 7 Mar 2026.
Builds on
57 of 69 references.
Closest ancestors · 12
Same direction, newest first
- 2026MIND: Benchmarking Memory Consistency and Action Control in World Models8 Feb
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
- 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
- 2025BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generation28 Nov
- 2025Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulation25 Nov
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
- 2024WorldSimBench: Towards Video Generation Models as World Simulators23 Oct
- 2024Genie: Generative Interactive Environments23 Feb
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
Other references · 45
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2025SAM 3: Segment Anything with Conceptscited by 835
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
33 more
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2025Continuous 3D Perception Model with Persistent Statecited by 527
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
- 2024Navigation World Modelscited by 262
- 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
- 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
- 2025History-Guided Video Diffusioncited by 165
- 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
- 2024In-Context LoRA for Diffusion Transformerscited by 155
- 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
- 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
- 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
- 2024CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Modelscited by 99
- 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 97
- 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
- 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
- 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
- 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
- 2025Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compressioncited by 72
- 2025PAN: A World Model for General, Actionable, and Long-Horizon World Simulationcited by 36
- 2025Spatia: Video Generation with Updatable Spatial Memorycited by 33
- 2025Whole-Body Conditioned Egocentric Video Predictioncited by 30
- 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 20
- 2025Yan: Foundational Interactive Video Generationcited by 15
- 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
- 2026Code2Worlds: Empowering Coding LLMs for 4D World Generationcited by 12
- 2026Flow Equivariant World Models: Memory for Partially Observed Dynamic Environmentscited by 6
- 2026GeoWorld: Geometric World Modelscited by 6
Built on by
Citing papers per year: 2026: 20.
20 papers.
Sort:
By area:
- 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generationcited by 11
- 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Modelscited by 10
- 2026Current World Models Lack a Persistent State Corecited by 8
- 2026Geometry-Aware Implicit Memory for Video World Modelscited by 7
- 2026Latent Spatial Memory for Video World Modelscited by 7
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 4
- 2026WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memorycited by 4
- 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoningcited by 3
- 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memorycited by 2
- 2026UniMesh: Unifying 3D Mesh Understanding and Generationcited by 1
- 2026FlashSign: Pose-Free Guidance for Efficient Sign Language Video Generationcited by 0
8 more
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
- 2026Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioningcited by 0
- 2026Can 4D Foundation Models Remember?cited by 0
- 2026Code Plans, Diffusion Renders: Open-Ended Generative World Modelingcited by 0
- 2026DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Timecited by 0
- 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Modelscited by 0
- 20264Director: Controlling Video World Models with Rigid 3D Geometrycited by 0
- 2026World Observer: Joint Actor-Observer Generation for Persistent World Modelingcited by 0
- 2026World Observer: Joint Actor-Observer Generation for Persistent World Modeling1 Oct 2026
- 20264Director: Controlling Video World Models with Rigid 3D Geometry1 Oct 2026
- 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Models29 Sep 2026
- 2026DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Time28 Sep 2026
- 2026Code Plans, Diffusion Renders: Open-Ended Generative World Modeling22 Sep 2026
- 2026Can 4D Foundation Models Remember?17 Sep 2026
- 2026Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioning1 Sep 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory2 Jul 2026
- 2026Current World Models Lack a Persistent State Core18 Jun 2026
- 2026Latent Spatial Memory for Video World Models8 Jun 2026
- 2026Echo-Memory: A Controlled Study of Memory in Action World Models8 Jun 2026
8 more
- 2026Geometry-Aware Implicit Memory for Video World Models1 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memory29 May 2026
- 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models18 May 2026
- 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning29 Apr 2026
- 2026UniMesh: Unifying 3D Mesh Understanding and Generation19 Apr 2026
- 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generation1 Apr 2026
- 2026FlashSign: Pose-Free Guidance for Efficient Sign Language Video Generation30 Mar 2026
- 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Modelscited by 10
- 2026Current World Models Lack a Persistent State Corecited by 8
- 2026Geometry-Aware Implicit Memory for Video World Modelscited by 7
- 2026Latent Spatial Memory for Video World Modelscited by 7
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 4
- 2026WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memorycited by 4
- 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoningcited by 3
- 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memorycited by 2
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
- 2026Code Plans, Diffusion Renders: Open-Ended Generative World Modelingcited by 0
- 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Modelscited by 0
12 most cited of 14 All 14 (JSON)
- 2026World Observer: Joint Actor-Observer Generation for Persistent World Modeling1 Oct 2026
- 20264Director: Controlling Video World Models with Rigid 3D Geometry1 Oct 2026
- 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Models29 Sep 2026
- 2026Code Plans, Diffusion Renders: Open-Ended Generative World Modeling22 Sep 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory2 Jul 2026
- 2026Current World Models Lack a Persistent State Core18 Jun 2026
- 2026Latent Spatial Memory for Video World Models8 Jun 2026
- 2026Echo-Memory: A Controlled Study of Memory in Action World Models8 Jun 2026
- 2026Geometry-Aware Implicit Memory for Video World Models1 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memory29 May 2026
12 newest of 14 All 14 (JSON)
- 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generationcited by 11
- 2026UniMesh: Unifying 3D Mesh Understanding and Generationcited by 1
- 2026FlashSign: Pose-Free Guidance for Efficient Sign Language Video Generationcited by 0
- 2026Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioningcited by 0
- 2026Can 4D Foundation Models Remember?cited by 0
- 2026DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Timecited by 0
- 2026DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Time28 Sep 2026
- 2026Can 4D Foundation Models Remember?17 Sep 2026
- 2026Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioning1 Sep 2026
- 2026UniMesh: Unifying 3D Mesh Understanding and Generation19 Apr 2026
- 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generation1 Apr 2026
- 2026FlashSign: Pose-Free Guidance for Efficient Sign Language Video Generation30 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Pixel Histories: World Models with Persistent 3D State
- 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Models
- 2026World in World: Explore the World with World Models
- 2026MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
- 2026AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video
- 2025Learning 3D Persistent Embodied World Models
- 2026LIVE: Long-horizon Interactive Video World Modeling
- 2026Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistency