Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perception
Not on the map. Why?
Abstract (arXiv)
4D content generation aims to create dynamically evolving 3D content that responds to specific input objects such as images or 3D representations. Current approaches typically incorporate physical priors to animate 3D representations, but these methods suffer from significant limitations: they not only require users lacking physics expertise to manually specify material properties but also struggle to effectively handle the generation of multi-material composite objects. To address these challenges, we propose Phys4DGen, a novel 4D generation framework that integrates multi-material composition perception with physical simulation. The framework achieves automated, physically plausible 4D generation through three innovative modules: first, the 3D Material Grouping module partitions heterogeneous material regions on 3D representations' surfaces via semantic segmentation; second, the Internal Physical Structure Discovery module constructs the mechanical structure of object interiors; finally, we distill physical prior knowledge from multimodal large language models to enable rapid and automatic material properties identification for both objects' surfaces and interiors. Experiments on both synthetic and real-world datasets demonstrate that Phys4DGen can generate high-fidelity 4D content with physical realism in open-world scenarios, significantly outperforming state-of-the-art methods.
Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 48 of its 60 references that are in the map. First posted on arXiv 25 Nov 2024.
Builds on
48 of 60 references.
References · 48
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023GPT-4 Technical Reportcited by 18,396
- 2023Segment Anythingcited by 8,234
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
36 more
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
- 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
- 2020U2-Net: Going Deeper with Nested U-Structure for Salient Object Detectioncited by 412
- 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
- 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
- 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
- 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 307
- 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
- 2023Text-To-4D Dynamic Scene Generationcited by 205
- 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 194
- 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 191
- 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
- 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
- 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
- 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
- 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 154
- 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 144
- 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 144
- 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 135
- 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 125
- 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
- 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 115
- 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
- 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
- 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 100
- 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 97
- 2023A Unified Approach for Text- and Image-guided 4D Scene Generationcited by 82
- 2024Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusioncited by 80
- 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 77
- 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
- 2024GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulationcited by 49
- 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
- 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generationcited by 32
- 2024Phy124: Fast Physics-Driven 4D Content Generation from a Single Imagecited by 23
Built on by
Citing papers per year: 2025: 3; 2026: 20.
23 papers.
Sort:
By area:
- 2025Puppeteer: Rig and Animate Your 3D Modelscited by 43
- 2026Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goalscited by 9
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 6
- 2026FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimizationcited by 4
- 2026NeuROK: Generative 4D Neural Object Kinematicscited by 3
- 2026MotionPhysics: Learnable Motion Distillation for Text-Guided Simulationcited by 2
- 2026OptiWorld: Optimal Control for Video World Generation under Physical Constraintscited by 2
- 2026CP4D: Compositional Physics-aware 4D Scene Generationcited by 2
- 2026OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidancecited by 1
- 2026Physics-Informed Video Generation via Mixture-of-Experts Latent Alignmentcited by 1
- 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolutioncited by 1
- 2026Physics-Grounded Fluid Video Generation with a Simulation Dataset and Dual-Stream Optical-Flow Supervisioncited by 1
11 more
- 2026Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decompositioncited by 1
- 2025Disentangled Geometric Alignment with Adaptive Contrastive Perturbation for Reliable Domain Transfercited by 0
- 2025Dynamic Epsilon Scheduling: A Multi-Factor Adaptive Perturbation Budget for Adversarial Trainingcited by 0
- 2026ReconMIL: Synergizing Latent Space Reconstruction with Bi-Stream Mamba for Whole Slide Image Analysiscited by 0
- 20263DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulationcited by 0
- 2026Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Spacecited by 0
- 2026Vision Meets WiFi: Physics-Grounded Estimation of Volumetric Mechanical Propertiescited by 0
- 2026Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videoscited by 0
- 2026PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generationcited by 0
- 2026PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Imagecited by 0
- 2026NeuIDO: Neural Intrinsic Dynamics Operator for Physics-Informed 4D World Modelscited by 0
- 2026NeuIDO: Neural Intrinsic Dynamics Operator for Physics-Informed 4D World Models21 Sep 2026
- 2026PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Image16 Sep 2026
- 2026PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generation8 Sep 2026
- 2026Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videos31 Aug 2026
- 2026Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decomposition24 Aug 2026
- 2026Vision Meets WiFi: Physics-Grounded Estimation of Volumetric Mechanical Properties7 Aug 2026
- 2026Physics-Grounded Fluid Video Generation with a Simulation Dataset and Dual-Stream Optical-Flow Supervision28 Jul 2026
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenes30 Jun 2026
- 2026Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Space24 Jun 2026
- 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolution16 Jun 2026
- 2026CP4D: Compositional Physics-aware 4D Scene Generation8 Jun 2026
- 2026Physics-Informed Video Generation via Mixture-of-Experts Latent Alignment3 Jun 2026
11 more
- 2026OptiWorld: Optimal Control for Video World Generation under Physical Constraints30 May 2026
- 2026NeuROK: Generative 4D Neural Object Kinematics28 May 2026
- 20263DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulation16 May 2026
- 2026ReconMIL: Synergizing Latent Space Reconstruction with Bi-Stream Mamba for Whole Slide Image Analysis20 Mar 2026
- 2026OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidance19 Mar 2026
- 2026FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimization2 Feb 2026
- 2026Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goals9 Jan 2026
- 2026MotionPhysics: Learnable Motion Distillation for Text-Guided Simulation1 Jan 2026
- 2025Puppeteer: Rig and Animate Your 3D Models14 Aug 2025
- 2025Dynamic Epsilon Scheduling: A Multi-Factor Adaptive Perturbation Budget for Adversarial Training3 Jun 2025
- 2025Disentangled Geometric Alignment with Adaptive Contrastive Perturbation for Reliable Domain Transfer21 May 2025
- 2025Puppeteer: Rig and Animate Your 3D Modelscited by 43
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 6
- 2026FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimizationcited by 4
- 2026NeuROK: Generative 4D Neural Object Kinematicscited by 3
- 2026MotionPhysics: Learnable Motion Distillation for Text-Guided Simulationcited by 2
- 2026CP4D: Compositional Physics-aware 4D Scene Generationcited by 2
- 2026OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidancecited by 1
- 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolutioncited by 1
- 2026Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decompositioncited by 1
- 2025Disentangled Geometric Alignment with Adaptive Contrastive Perturbation for Reliable Domain Transfercited by 0
- 2025Dynamic Epsilon Scheduling: A Multi-Factor Adaptive Perturbation Budget for Adversarial Trainingcited by 0
- 20263DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulationcited by 0
12 most cited of 17 All 17 (JSON)
- 2026PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Image16 Sep 2026
- 2026PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generation8 Sep 2026
- 2026Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videos31 Aug 2026
- 2026Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decomposition24 Aug 2026
- 2026Vision Meets WiFi: Physics-Grounded Estimation of Volumetric Mechanical Properties7 Aug 2026
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenes30 Jun 2026
- 2026Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Space24 Jun 2026
- 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolution16 Jun 2026
- 2026CP4D: Compositional Physics-aware 4D Scene Generation8 Jun 2026
- 2026NeuROK: Generative 4D Neural Object Kinematics28 May 2026
- 20263DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulation16 May 2026
- 2026OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidance19 Mar 2026
12 newest of 17 All 17 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2025PhysX-3D: Physical-Grounded 3D Asset Generation
- 2024Precise-Physics Driven Text-to-3D Generation
- 2026PhysForge: Material-Controllable 3D Reconstruction with Separable Components
- 2026PhysReal: Learning Real-World Deformable Object Physics via Hybrid Constitutive Modeling
- 2026PhysMAS: Physics-Grounded Multi-Agent Synthesis of Compositional 4D Gaussians
- 2024Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials
- 2024MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets
- 2025MatPedia: A Universal Generative Foundation for High-Fidelity Material Synthesis