# LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models

arXiv: https://arxiv.org/abs/2603.07145 · HTML: https://mlsanity.com/paper/2603.07145/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 57 of its 69 references that are in the map.
- First posted on arXiv 7 Mar 2026.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 30 of 57)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [MapAnything: Universal Feed-Forward Metric 3D Reconstruction](https://mlsanity.com/paper/2509.13414/) (2025)
- [Navigation World Models](https://mlsanity.com/paper/2412.03572/) (2024)
- [MagicDrive: Street View Generation with Diverse 3D Geometry Control](https://mlsanity.com/paper/2310.02601/) (2023)
- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [Rolling Forcing: Autoregressive Long Video Diffusion in Real Time](https://mlsanity.com/paper/2509.25161/) (2025)
- [Self-Forcing++: Towards Minute-Scale High-Quality Video Generation](https://mlsanity.com/paper/2510.02283/) (2025)
- [Matrix-game 2.0: An open-source real-time and streaming interactive world model](https://mlsanity.com/paper/2508.13009/) (2025)
- [History-Guided Video Diffusion](https://mlsanity.com/paper/2502.06764/) (2025)

## Built on by (most cited first, 28 of 28)

- [Latent Spatial Memory for Video World Models](https://mlsanity.com/paper/2606.09828/) (2026)
- [EgoSim: Egocentric World Simulator for Embodied Interaction Generation](https://mlsanity.com/paper/2604.01001/) (2026)
- [Geometry-Aware Implicit Memory for Video World Models](https://mlsanity.com/paper/2606.02436/) (2026)
- [Current World Models Lack a Persistent State Core](https://mlsanity.com/paper/2606.20545/) (2026)
- [Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models](https://mlsanity.com/paper/2605.18601/) (2026)
- [Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends](https://mlsanity.com/paper/2606.01164/) (2026)
- [Echo-Memory: A Controlled Study of Memory in Action World Models](https://mlsanity.com/paper/2606.09803/) (2026)
- [WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory](https://mlsanity.com/paper/2607.02517/) (2026)
- [World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning](https://mlsanity.com/paper/2604.26934/) (2026)
- [FlashAR: Efficient Post-Training Acceleration for Autoregressive Image Generation](https://mlsanity.com/paper/2605.09430/) (2026)
- [UniMesh: Unifying 3D Mesh Understanding and Generation](https://mlsanity.com/paper/2604.17472/) (2026)
- [DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memory](https://mlsanity.com/paper/2605.31336/) (2026)
- [FlashSign: Pose-Free Guidance for Efficient Sign Language Video Generation](https://mlsanity.com/paper/2603.27915/) (2026)
- [From Generation to Simulation: How Far Are World Models from Being True Simulators?](https://mlsanity.com/paper/2608.23070/) (2026)
- [Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioning](https://mlsanity.com/paper/2609.00656/) (2026)
- [Can 4D Foundation Models Remember?](https://mlsanity.com/paper/2609.20819/) (2026)
- [Code Plans, Diffusion Renders: Open-Ended Generative World Modeling](https://mlsanity.com/paper/2609.26458/) (2026)
- [DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Time](https://mlsanity.com/paper/2609.35704/) (2026)
- [Honeycomb: Constant-Size Scene Memory Representation for Video World Models](https://mlsanity.com/paper/2609.37690/) (2026)
- [4Director: Controlling Video World Models with Rigid 3D Geometry](https://mlsanity.com/paper/2610.02160/) (2026)
- [World Observer: Joint Actor-Observer Generation for Persistent World Modeling](https://mlsanity.com/paper/2610.02162/) (2026)
- [EVEWorld: Physical Evolution Supervision for Embodied World Models](https://mlsanity.com/paper/2610.03374/) (2026)
- [Kepler4D: Controllable Future Video Generation via 4D Scene State Evolution](https://mlsanity.com/paper/2610.04152/) (2026)
- [4LC: 4D Latent Cache for Efficient Conditioning in Dynamic World Modeling](https://mlsanity.com/paper/2610.17625/) (2026)
- [Grid or Nodes? Probing Object Memory in World Models Under Partial Observability](https://mlsanity.com/paper/2610.18637/) (2026)
- [WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models](https://mlsanity.com/paper/2610.19063/) (2026)
- [StateCraft: Augmenting Video World Models with Abstract-State Reasoning](https://mlsanity.com/paper/2610.19795/) (2026)
- [PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents](https://mlsanity.com/paper/2610.20240/) (2026)

## Abstract (arXiv)

Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control. However, they implicitly assume that the world only evolves within the observer's field of view. Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime. In this work, we identify and formalize this overlooked limitation as the "out-of-sight dynamics" problem, which impedes video world models from representing a continuously evolving world. To address this issue, we propose LiveWorld, a novel framework that extends video world models to support persistent world evolution. Instead of treating the world as static observational memory, LiveWorld models a persistent global state composed of a static 3D background and dynamic entities that continue evolving even when unobserved. To maintain these unseen dynamics, LiveWorld introduces a monitor-based mechanism that autonomously simulates the temporal progression of active entities and synchronizes their evolved states upon revisiting, ensuring spatially coherent rendering. For evaluation, we further introduce LiveBench, a dedicated benchmark for the task of maintaining out-of-sight dynamics. Extensive experiments show that LiveWorld enables persistent event evolution and long-term scene consistency, bridging the gap between existing 2D observation-based memory and true 4D dynamic world simulation. The baseline and benchmark will be publicly available at https://zichengduan.github.io/LiveWorld/index.html.