PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generation
Not on the map. Why?
Abstract (arXiv)
We introduce PerpetualWonder, a hybrid generative simulator that enables long-horizon, action-conditioned 4D scene generation from a single image. Current works fail at this task because their physical state is decoupled from their visual representation, which prevents generative refinements to update the underlying physics for subsequent interactions. PerpetualWonder solves this by introducing the first true closed-loop system. It features a novel unified representation that creates a bidirectional link between the physical state and visual primitives, allowing generative refinements to correct both the dynamics and appearance. It also introduces a robust update mechanism that gathers supervision from multiple viewpoints to resolve optimization ambiguity. Experiments demonstrate that from a single image, PerpetualWonder can successfully simulate complex, multi-step interactions from long-horizon actions, maintaining physical plausibility and visual consistency.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 56 of its 64 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
56 of 64 references.
References · 56
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2024GPT-4o System Cardcited by 4,467
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
- 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 470
- 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 387
38 more
- 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 368
- 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
- 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 333
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
- 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
- 2025Video models are zero-shot learners and reasonerscited by 208
- 2023Text-To-4D Dynamic Scene Generationcited by 204
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
- 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
- 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
- 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
- 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
- 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
- 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
- 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
- 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 161
- 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
- 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
- 2023WonderJourney: Going from Anywhere to Everywherecited by 157
- 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 143
- 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
- 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 114
- 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
- 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesiscited by 99
- 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
- 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
- 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 87
- 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
- 2025CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generationcited by 73
- 2024Controlling Space and Time with Diffusion Modelscited by 65
- 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 65
- 2024SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splattingcited by 63
- 2025FreeTimeGS: Free Gaussian Primitives at Anytime and Anywhere for Dynamic Scene Reconstructioncited by 53
- 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 51
- 2024GenXD: Generating Any 3D and 4D Scenescited by 49
- 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
- 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 43
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agentscited by 5
- 2026DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videoscited by 1
- 2026LivingWorld: Interactive 4D World Generation with Environmental Dynamicscited by 0
- 20263DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulationcited by 0
- 2026Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Spacecited by 0
- 2026MotionInsight: Diagnosing Object Motion Deficiencies in Generated Videoscited by 0
- 2026World Motion Models: Flexible Sequence Modeling of SE(3) Trajectoriescited by 0
- 2026Fluid-Gen-Zero: Grounding Pretrained Video Generators in Physics without Trainingcited by 0
- 2026Fluid-Gen-Zero: Grounding Pretrained Video Generators in Physics without Training7 Oct 2026
- 2026World Motion Models: Flexible Sequence Modeling of SE(3) Trajectories1 Oct 2026
- 2026MotionInsight: Diagnosing Object Motion Deficiencies in Generated Videos29 Sep 2026
- 2026Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Space24 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 20263DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulation16 May 2026
- 2026DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos10 May 2026
- 2026LivingWorld: Interactive 4D World Generation with Environmental Dynamics2 Apr 2026
- 2026SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agents9 Mar 2026