INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
Building world models with spatial consistency and real-time interactivity remains a fundamental challenge in computer vision. Current video generation paradigms often struggle with a lack of spatial persistence and insufficient visual realism, making it difficult to support seamless navigation in complex environments. To address these challenges, we propose INSPATIO-WORLD, a novel real-time framework capable of recovering and generating high-fidelity, dynamic interactive scenes from a single reference video. At the core of our approach is a Spatiotemporal Autoregressive (STAR) architecture, which enables consistent and controllable scene evolution through two tightly coupled components: Implicit Spatiotemporal Cache aggregates reference and historical observations into a latent world representation, ensuring global consistency during long-horizon navigation; Explicit Spatial Constraint Module enforces geometric structure and translates user interactions into precise and physically plausible camera trajectories. Furthermore, we introduce Joint Distribution Matching Distillation (JDMD). By using real-world data distributions as a regularizing guide, JDMD effectively overcomes the fidelity degradation typically caused by over-reliance on synthetic data. Extensive experiments demonstrate that INSPATIO-WORLD significantly outperforms existing state-of-the-art (SOTA) models in spatial consistency and interaction precision, ranking first among real-time interactive methods on the WorldScore-Dynamic benchmark, and establishing a practical pipeline for navigating 4D environments reconstructed from monocular videos.
Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 93 of its 94 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
93 of 94 references.
Closest ancestors · 13
Same direction, newest first
- 2026InSpatio-WorldFM: An Open-Source Real-Time Generative Frame Model12 Mar
- 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory2 Feb
- 2026Advancing Open-source World Models28 Jan
- 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos1 Jan
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
- 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Prediction25 Sep
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025MAGI-1: Autoregressive Video Generation at Scale19 May
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2024Open-Sora: Democratizing Efficient Video Production for All29 Dec
- 2024Genie: Generative Interactive Environments23 Feb
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
1 more
Other references · 80
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Video Diffusion Modelscited by 2,280
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
- 2024Autoregressive Image Generation without Vector Quantizationcited by 644
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
38 more
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
- 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
- 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
- 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
- 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2023Photorealistic Video Generation with Diffusion Modelscited by 304
- 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
- 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
- 2024Navigation World Modelscited by 262
- 2019Scaling Autoregressive Video Modelscited by 189
- 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
- 2024Autoregressive Video Generation without Vector Quantizationcited by 180
- 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 163
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
- 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
- 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
- 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 142
- 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 134
- 2025Test-Time Training Done Rightcited by 121
- 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
- 2024Training-free Camera Control for Video Generationcited by 119
- 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
- 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
- 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 115
- 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
- 2025Long Context Tuning for Video Generationcited by 110
- 2024MotionClone: Training-Free Motion Cloning for Controllable Video Generationcited by 104
- 2024Rolling Diffusion Modelscited by 102
- 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
- 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
- 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 97
50 most cited of 80 All 80 (JSON)
Built on by
Citing papers per year: 2026: 18.
18 papers.
Sort:
By area:
- 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
- 2026Wonder: Video World Model Done Bettercited by 13
- 2026Current World Models Lack a Persistent State Corecited by 8
- 2026Geometry-Aware Implicit Memory for Video World Modelscited by 7
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registerscited by 5
- 2026Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolutioncited by 4
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?cited by 4
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worldscited by 3
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivitycited by 2
- 2026World-Action Models for Robot Learning and Control: A Surveycited by 1
6 more
- 2026Precise Editing and Flexible Referencing for Interactable Worldscited by 1
- 2026Oneira: From Open-Ended Generation to Open-World Interaction in Video World Modelscited by 1
- 2026Robust Dreamer: Deviation-Aware Latent Gaussian Memory for Action-Controlled AR Video Generationcited by 0
- 2026World in World: Explore the World with World Modelscited by 0
- 2026WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizoncited by 0
- 2026Exo2EgoHOI: Hand-Object-Interaction Aware Exocentric-to-Egocentric Video Generationcited by 0
- 2026Oneira: From Open-Ended Generation to Open-World Interaction in Video World Models1 Oct 2026
- 2026Exo2EgoHOI: Hand-Object-Interaction Aware Exocentric-to-Egocentric Video Generation29 Sep 2026
- 2026WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon28 Sep 2026
- 2026Precise Editing and Flexible Referencing for Interactable Worlds28 Sep 2026
- 2026World-Action Models for Robot Learning and Control: A Survey13 Sep 2026
- 2026World in World: Explore the World with World Models10 Sep 2026
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worlds17 Aug 2026
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity3 Aug 2026
- 2026Wonder: Video World Model Done Better28 Jul 2026
- 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers23 Jul 2026
- 2026Current World Models Lack a Persistent State Core18 Jun 2026
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?9 Jun 2026
6 more
- 2026Geometry-Aware Implicit Memory for Video World Models1 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026Robust Dreamer: Deviation-Aware Latent Gaussian Memory for Action-Controlled AR Video Generation29 May 2026
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation25 May 2026
- 2026Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution25 May 2026
- 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer14 May 2026
- 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
- 2026Wonder: Video World Model Done Bettercited by 13
- 2026Current World Models Lack a Persistent State Corecited by 8
- 2026Geometry-Aware Implicit Memory for Video World Modelscited by 7
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registerscited by 5
- 2026Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolutioncited by 4
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?cited by 4
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worldscited by 3
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivitycited by 2
- 2026World-Action Models for Robot Learning and Control: A Surveycited by 1
12 most cited of 16 All 16 (JSON)
- 2026Oneira: From Open-Ended Generation to Open-World Interaction in Video World Models1 Oct 2026
- 2026WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon28 Sep 2026
- 2026Precise Editing and Flexible Referencing for Interactable Worlds28 Sep 2026
- 2026World-Action Models for Robot Learning and Control: A Survey13 Sep 2026
- 2026World in World: Explore the World with World Models10 Sep 2026
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worlds17 Aug 2026
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity3 Aug 2026
- 2026Wonder: Video World Model Done Better28 Jul 2026
- 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers23 Jul 2026
- 2026Current World Models Lack a Persistent State Core18 Jun 2026
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?9 Jun 2026
- 2026Geometry-Aware Implicit Memory for Video World Models1 Jun 2026
12 newest of 16 All 16 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold
- 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models
- 2026Beyond Pixel Histories: World Models with Persistent 3D State
- 2026Sekai2: From World Exploration to Interactive World Modeling
- 2025TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model
- 2026BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression
- 2026WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation
- 2025IC-World: In-Context Generation for Shared World Modeling