mlsanityData to 14 Oct 2026

PhysGen3D: Crafting a Miniature Interactive World from a Single Image

arXiv 2503.20746 ↗First posted 26 Mar 2025cs.CVcited by 66

Not on the map. Why?

Abstract (arXiv)

Envisioning physically plausible outcomes from a single image requires a deep understanding of the world's dynamics. To address this, we introduce PhysGen3D, a novel framework that transforms a single image into an amodal, camera-centric, interactive 3D scene. By combining advanced image-based geometric and semantic understanding with physics-based simulation, PhysGen3D creates an interactive 3D world from a static image, enabling us to "imagine" and simulate future scenarios based on user input. At its core, PhysGen3D estimates 3D shapes, poses, physical and lighting properties of objects, thereby capturing essential physical attributes that drive realistic object interactions. This framework allows users to specify precise initial conditions, such as object speed or material properties, for enhanced control over generated video outcomes. We evaluate PhysGen3D's performance against closed-source state-of-the-art (SOTA) image-to-video models, including Pika, Kling, and Gen-3, showing PhysGen3D's capacity to generate videos with realistic physics while offering greater flexibility and fine-grained control. Our results show that PhysGen3D achieves a unique balance of photorealism, physical plausibility, and user-driven interactivity, opening new possibilities for generating dynamic, physics-grounded video from an image.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Builds on 76 of its 109 references that are in the map. First posted on arXiv 26 Mar 2025.

Builds on

76 of 109 references.

References · 76

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023Segment Anythingcited by 8,234
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  12. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
38 more
  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  3. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  4. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  5. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  6. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  7. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  8. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  9. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  10. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
  11. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  12. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  13. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  14. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  15. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  16. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
  17. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
  18. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 350
  19. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  20. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
  21. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 297
  22. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 285
  23. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  24. 2019DiffTaichi: Differentiable Programming for Physical Simulationcited by 258
  25. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 255
  26. 2022Diffusion Probabilistic Modeling for Video Generationcited by 238
  27. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 233
  28. 2022Thin-Plate Spline Motion Model for Image Animationcited by 210
  29. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 191
  30. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
  31. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  32. 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 154
  33. 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
  34. 2020Unpaired Motion Style Transfer from Video to Animationcited by 94
  35. 2023Generative Image Dynamicscited by 94
  36. 2023TrailBlazer: Trajectory Control for Diffusion-Based Video Generationcited by 91
  37. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 89
  38. 2023LLM-grounded Video Diffusion Modelscited by 83

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 41.

66 papers.

Sort:

By area:

  1. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  2. 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 53
  3. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 45
  4. 2025PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generationcited by 41
  5. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 41
  6. 2025PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Imagecited by 37
  7. 2025Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactionscited by 30
  8. 2025Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixelscited by 24
  9. 2026RealWonder: Real-Time Physical Action-Conditioned Video Generationcited by 22
  10. 2025PAT3D: Physics-Augmented Text-to-3D Scene Generationcited by 16
  11. 2025HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Videocited by 14
  12. 2026Physical Simulator In-the-Loop Video Generationcited by 14
38 more
  1. 2026PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objectscited by 11
  2. 2025PhysGM: Large Physical Gaussian Model for Feed-Forward 4D Synthesiscited by 10
  3. 2025PhysChoreo: Physics-Controllable Video Generation with Part-Aware Semantic Groundingcited by 9
  4. 2026PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generationcited by 9
  5. 2025SIMPACT: Simulation-Enabled Action Planning using Vision-Language Modelscited by 8
  6. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 8
  7. 2025Robot Learning from Any Imagescited by 7
  8. 2025Is Single-View Mesh Reconstruction Ready for Robotics?cited by 6
  9. 2025Product of Experts for Visual Generationcited by 6
  10. 2026Learning Physics-Grounded 4D Dynamics with Neural Gaussian Force Fieldscited by 6
  11. 2026Engine-Native Editable 3D World Reconstruction with Objects and Lightingcited by 6
  12. 2025SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materialscited by 5
  13. 2025ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editingcited by 5
  14. 2025SeeU: Seeing the Unseen World via 4D Dynamics-aware Generationcited by 5
  15. 2026Choreographing a World of Dynamic Objectscited by 5
  16. 2026SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Imagecited by 5
  17. 2026PhysAgent: Automating Physics-Based 4D Synthesis via Trajectory-Grounded Multi-Agent Feedbackcited by 5
  18. 2026GS-Agent: Creating 4D Physical Worlds With Generative Simulationcited by 5
  19. 2026FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimizationcited by 4
  20. 2026PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignmentcited by 4
  21. 2026PhysOmni: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interactioncited by 4
  22. 2025VDAWorld: World Modelling via VLM-Directed Abstraction and Simulationcited by 3
  23. 2025Over++: Generative Video Compositing for Layer Interaction Effectscited by 3
  24. 2026NeuROK: Generative 4D Neural Object Kinematicscited by 3
  25. 2025Learning to Generate Rigid Body Interactions with Video Diffusion Modelscited by 2
  26. 2026MoRight: Motion Control Done Rightcited by 2
  27. 2026Envisioning the Future, One Step at a Timecited by 2
  28. 2026DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videoscited by 2
  29. 2026Physically Viable World Models: A Case for Query-Conditioned Embodied AIcited by 2
  30. 2026CP4D: Compositional Physics-aware 4D Scene Generationcited by 2
  31. 2026PhysiFormer: Learning to Simulate Mechanics in World Spacecited by 2
  32. 2026PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generationcited by 2
  33. 2026Principia: Relational Physics Tests for Video Modelscited by 2
  34. 2026TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Imagecited by 2
  35. 2026PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Controlcited by 2
  36. 2025NeoWorld: Neural Simulation of Explorable Virtual Worlds via Progressive 3D Unfoldingcited by 1
  37. 2025Ctrl-VI: Controllable Video Synthesis via Variational Inferencecited by 1
  38. 2025Object Reconstruction under Occlusion with Generative Priors and Contact-induced Constraintscited by 1

50 most cited of 66 All 66 (JSON)