mlsanityData to 7 Oct 2026

WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actions

arXiv 2505.18151 ↗First posted 23 May 2025cs.GR cs.AI cs.CVcited by 43

Not on the map. Why?

Abstract (arXiv)

WonderPlay is a novel framework integrating physics simulation with video generation for generating action-conditioned dynamic 3D scenes from a single image. While prior works are restricted to rigid body or simple elastic dynamics, WonderPlay features a hybrid generative simulator to synthesize a wide range of 3D dynamics. The hybrid generative simulator first uses a physics solver to simulate coarse 3D dynamics, which subsequently conditions a video generator to produce a video with finer, more realistic motion. The generated video is then used to update the simulated dynamic 3D scene, closing the loop between the physics solver and the video generator. This approach enables intuitive user control to be combined with the accurate dynamics of physics-based simulators and the expressivity of diffusion-based video generators. Experimental results demonstrate that WonderPlay enables users to interact with various scenes of diverse content, including cloth, sand, snow, liquid, smoke, elastic, and rigid bodies -- all using a single image input. Code will be made public. Project website: https://kyleleey.github.io/WonderPlay/

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 63 of its 75 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

63 of 75 references.

References · 63

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2023Segment Anythingcited by 8,158
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  7. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  9. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  10. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  12. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
38 more
  1. 2018World Modelscited by 1,444
  2. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  3. 2024Genie: Generative Interactive Environmentscited by 706
  4. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  5. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  6. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  7. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 368
  8. 2024Diffusion Models Are Real-Time Game Enginescited by 282
  9. 2024Navigation World Modelscited by 262
  10. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  11. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  12. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  13. 2023Text-To-4D Dynamic Scene Generationcited by 204
  14. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  15. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  16. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
  17. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  18. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  19. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  20. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
  21. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 163
  22. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 161
  23. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  24. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  25. 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 145
  26. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  27. 2024GameGen-X: Interactive Open-world Game Video Generationcited by 124
  28. 2024Training-free Camera Control for Video Generationcited by 119
  29. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 115
  30. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 114
  31. 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 113
  32. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesiscited by 99
  33. 2023Generative Image Dynamicscited by 93
  34. 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 92
  35. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 87
  36. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  37. 2024Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusioncited by 76
  38. 2024SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generationcited by 68

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 33.

43 papers.

Sort:

By area:

  1. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  2. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 74
  3. 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 51
  4. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 40
  5. 2025PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generationcited by 39
  6. 2025Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixelscited by 22
  7. 2026RealWonder: Real-Time Physical Action-Conditioned Video Generationcited by 18
  8. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  9. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
  10. 2026Physical Simulator In-the-Loop Video Generationcited by 12
  11. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Modelscited by 12
  12. 2025PhysChoreo: Physics-Controllable Video Generation with Part-Aware Semantic Groundingcited by 9
31 more
  1. 2026Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goalscited by 9
  2. 2026PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generationcited by 9
  3. 2026MemLearner: Learning to Query Context memory for Video World Modelscited by 8
  4. 2026SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agentscited by 5
  5. 2026PhyCo: Learning Controllable Physical Priors for Generative Motioncited by 5
  6. 2026Choreographing a World of Dynamic Objectscited by 4
  7. 2026SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-body Manipulationcited by 4
  8. 2026GS-Agent: Creating 4D Physical Worlds With Generative Simulationcited by 4
  9. 2025SeeU: Seeing the Unseen World via 4D Dynamics-aware Generationcited by 3
  10. 2026PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignmentcited by 3
  11. 2026Learning Explicit Physical Parameter Control and Benchmarking for Video Generationcited by 3
  12. 2025Learning to Generate Rigid Body Interactions with Video Diffusion Modelscited by 2
  13. 2026MoRight: Motion Control Done Rightcited by 2
  14. 2026Envisioning the Future, One Step at a Timecited by 2
  15. 2026NeuROK: Generative 4D Neural Object Kinematicscited by 2
  16. 2025Inferring Compositional 4D Scenes without Ever Seeing Onecited by 1
  17. 2026DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videoscited by 1
  18. 2026Physics-Informed Video Generation via Mixture-of-Experts Latent Alignmentcited by 1
  19. 2026UniPixie: Unified and Probabilistic 3D Physics Learning via Flow Matchingcited by 1
  20. 2026Streaming Video Generation with Streaming Force Controlcited by 1
  21. 2026NEXUS: Neural Energy Fields for Physically Consistent Contact-Rich 3D Object Dynamicscited by 1
  22. 2026LivingWorld: Interactive 4D World Generation with Environmental Dynamicscited by 0
  23. 20263DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulationcited by 0
  24. 2026Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Spacecited by 0
  25. 2026DR-GS: Physically-Based Deformable and Relightable 2D Gaussianscited by 0
  26. 2026DeforM: Reasoning-Guided Physics-Aware Video Generation via Spatial-Temporal Maskingcited by 0
  27. 2026SNF-Bench: Separating Static Drift from Natural Flow in Long-Horizon Fixed-Camera Video Generationcited by 0
  28. 2026PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generationcited by 0
  29. 2026NeuIDO: Neural Intrinsic Dynamics Operator for Physics-Informed 4D World Modelscited by 0
  30. 2026Audible World Models: Spatially Aware Sound Generation for 3D Worldscited by 0
  31. 2026Generative Cinematographer: Composing Camera and Object Motion in 3Dcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 20253D Scene Generation: A Survey
  2. 2024DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors
  3. 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Models
  4. 2024Phy124: Fast Physics-Driven 4D Content Generation from a Single Image
  5. 2025RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism
  6. 2025WonderTurbo: Generating Interactive 3D World in 0.72 Seconds
  7. 2024Text-to-3D Gaussian Splatting with Physics-Grounded Motion Generation
  8. 2024ViMo: Generating Motions from Casual Videos