mlsanityData to 14 Oct 2026

PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation

arXiv 2509.20358 ↗First posted 24 Sep 2025cs.CVcited by 41

On the map: Diffusion models → Video generation

Abstract (arXiv)

Existing video generation models excel at producing photo-realistic videos from text or images, but often lack physical plausibility and 3D controllability. To overcome these limitations, we introduce PhysCtrl, a novel framework for physics-grounded image-to-video generation with physical parameters and force control. At its core is a generative physics network that learns the distribution of physical dynamics across four materials (elastic, sand, plasticine, and rigid) via a diffusion model conditioned on physics parameters and applied forces. We represent physical dynamics as 3D point trajectories and train on a large-scale synthetic dataset of 550K animations generated by physics simulators. We enhance the diffusion model with a novel spatiotemporal attention block that emulates particle interactions and incorporates physics-based constraints during training to enforce physical plausibility. Experiments show that PhysCtrl generates realistic, physics-grounded motion trajectories which, when used to drive image-to-video models, yield high-fidelity, controllable videos that outperform existing methods in both visual quality and physical plausibility. Project Page: https://cwchenwang.github.io/physctrl

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 76 of its 102 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

76 of 102 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noise14 Jan
  2. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Control7 Jan
  3. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  4. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  5. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  6. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generation2 Apr
  7. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance21 Mar
  8. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  9. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation4 Dec
  10. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation28 Nov
  11. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  12. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generation30 Oct
5 more
  1. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors18 Oct
  2. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  4. 2022Human Motion Diffusion Model29 Sep
  5. 2022Video Diffusion Models7 Apr

Other references · 59

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2023Segment Anythingcited by 8,234
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  8. 2018Neural Ordinary Differential Equationscited by 3,779
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  10. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  11. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  12. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
38 more
  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  4. 2020Learning to Simulate Complex Physics with Graph Networkscited by 880
  5. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  6. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  7. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  8. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
  9. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  10. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
  11. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  12. 2023SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenescited by 347
  13. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 324
  14. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 319
  15. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 312
  16. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  17. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  18. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 191
  19. 2024VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Realitycited by 179
  20. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 179
  21. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
  22. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
  23. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  24. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  25. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  26. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
  27. 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 97
  28. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 87
  29. 2022RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networkscited by 72
  30. 20243DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generationcited by 71
  31. 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 66
  32. 2022CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphismcited by 57
  33. 2024AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulationcited by 54
  34. 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 53
  35. 2022Physics Informed Neural Fields for Smoke Reconstruction with Sparse Datacited by 52
  36. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  37. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 45
  38. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 42

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 36.

41 papers.

Sort:

By area:

  1. 2025What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewardscited by 25
  2. 2026Self-Refining Video Samplingcited by 15
  3. 2026Physical Simulator In-the-Loop Video Generationcited by 14
  4. 2026LOME: Learning Human-Object Manipulation with Action-Conditioned Egocentric World Modelcited by 13
  5. 2026NEWTON: Agentic Planning for Physically Grounded Video Generationcited by 12
  6. 2025PhysChoreo: Physics-Controllable Video Generation with Part-Aware Semantic Groundingcited by 9
  7. 2026VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine Systemcited by 8
  8. 2025Planning with Sketch-Guided Verification for Physics-Aware Video Generationcited by 6
  9. 2025Objects in Generated Videos Are Slower Than They Appear: Models Suffer Sub-Earth Gravity and Don't Know Galileo's Principle...for nowcited by 5
  10. 2026PhyCo: Learning Controllable Physical Priors for Generative Motioncited by 5
  11. 2026PhysAgent: Automating Physics-Based 4D Synthesis via Trajectory-Grounded Multi-Agent Feedbackcited by 5
  12. 2026GS-Agent: Creating 4D Physical Worlds With Generative Simulationcited by 5
29 more
  1. 2026PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignmentcited by 4
  2. 2026Next-Scale Autoregressive Models for Text-to-Motion Generationcited by 4
  3. 2026PhysOmni: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interactioncited by 4
  4. 2026LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generationcited by 4
  5. 2026WorldParticle: Unified World Simulation of Lagrangian Particle Dynamics via Transformercited by 3
  6. 2026Phys4D: Fine-Grained Physics-Consistent 4D Modeling from Video Diffusioncited by 2
  7. 2026Pretrained Video Models as Differentiable Physics Simulators for Urban Wind Flowscited by 2
  8. 2026Lighting-grounded Video Generation with Renderer-based Agent Reasoningcited by 2
  9. 2026Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generationcited by 2
  10. 2026PhysEditWorld: A Large-Scale Dataset Toward Physics-Editable World Modelscited by 2
  11. 2026PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generationcited by 2
  12. 2026Principia: Relational Physics Tests for Video Modelscited by 2
  13. 2026TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Imagecited by 2
  14. 2026Seeking Physics in Diffusion Noisecited by 1
  15. 2026UniPixie: Unified and Probabilistic 3D Physics Learning via Flow Matchingcited by 1
  16. 2026Streaming Video Generation with Streaming Force Controlcited by 1
  17. 2026NEXUS: Neural Energy Fields for Physically Consistent Contact-Rich 3D Object Dynamicscited by 1
  18. 2025A Study of the Design Space of Motion and Disocclusion Control in Video Generationcited by 0
  19. 2026Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMscited by 0
  20. 20263DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulationcited by 0
  21. 2026Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Spacecited by 0
  22. 2026PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generationcited by 0
  23. 2026SIFT: Self-Imagination Fine-Tuning for Physically Plausible Motion in Video Diffusion Modelscited by 0
  24. 2026Occlusion-Robust Multi-Object Decoupling for Physics-Based Robotic Interactioncited by 0
  25. 2026VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generationcited by 0
  26. 2026Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioningcited by 0
  27. 2026PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generationcited by 0
  28. 2026FracGen: Learning How Objects Stretch and Tear with Physics-Informed Video Generationcited by 0
  29. 2026World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generation
  2. 2025PhyMAGIC: Physical Motion-Aware Generative Inference with Confidence-guided LLM
  3. 2026Learning Explicit Physical Parameter Control and Benchmarking for Video Generation
  4. 2026PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models
  5. 2025PhysMaster: Mastering Physical Representation for Video Generation via Reinforcement Learning
  6. 2026PhyRPR: Training-Free Physics-Constrained Video Generation
  7. 2026PhysInOne: Visual Physics Learning and Reasoning in One Suite
  8. 2026PhyGround: Benchmarking Physical Reasoning in Generative World Models