mlsanityData to 14 Oct 2026

Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perception

arXiv 2411.16800 ↗First posted 25 Nov 2024cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

4D content generation aims to create dynamically evolving 3D content that responds to specific input objects such as images or 3D representations. Current approaches typically incorporate physical priors to animate 3D representations, but these methods suffer from significant limitations: they not only require users lacking physics expertise to manually specify material properties but also struggle to effectively handle the generation of multi-material composite objects. To address these challenges, we propose Phys4DGen, a novel 4D generation framework that integrates multi-material composition perception with physical simulation. The framework achieves automated, physically plausible 4D generation through three innovative modules: first, the 3D Material Grouping module partitions heterogeneous material regions on 3D representations' surfaces via semantic segmentation; second, the Internal Physical Structure Discovery module constructs the mechanical structure of object interiors; finally, we distill physical prior knowledge from multimodal large language models to enable rapid and automatic material properties identification for both objects' surfaces and interiors. Experiments on both synthetic and real-world datasets demonstrate that Phys4DGen can generate high-fidelity 4D content with physical realism in open-world scenarios, significantly outperforming state-of-the-art methods.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 48 of its 60 references that are in the map. First posted on arXiv 25 Nov 2024.

Builds on

48 of 60 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023Segment Anythingcited by 8,234
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  9. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  12. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
36 more
  1. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  2. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  3. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  4. 2020U2-Net: Going Deeper with Nested U-Structure for Salient Object Detectioncited by 412
  5. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  6. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  7. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
  8. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  9. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 307
  10. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  11. 2023Text-To-4D Dynamic Scene Generationcited by 205
  12. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 194
  13. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 191
  14. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
  15. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  16. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
  17. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  18. 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 154
  19. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 144
  20. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 144
  21. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 135
  22. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 125
  23. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  24. 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 115
  25. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  26. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  27. 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 100
  28. 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 97
  29. 2023A Unified Approach for Text- and Image-guided 4D Scene Generationcited by 82
  30. 2024Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusioncited by 80
  31. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 77
  32. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  33. 2024GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulationcited by 49
  34. 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
  35. 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generationcited by 32
  36. 2024Phy124: Fast Physics-Driven 4D Content Generation from a Single Imagecited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 20.

23 papers.

Sort:

By area:

  1. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 43
  2. 2026Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goalscited by 9
  3. 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 6
  4. 2026FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimizationcited by 4
  5. 2026NeuROK: Generative 4D Neural Object Kinematicscited by 3
  6. 2026MotionPhysics: Learnable Motion Distillation for Text-Guided Simulationcited by 2
  7. 2026OptiWorld: Optimal Control for Video World Generation under Physical Constraintscited by 2
  8. 2026CP4D: Compositional Physics-aware 4D Scene Generationcited by 2
  9. 2026OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidancecited by 1
  10. 2026Physics-Informed Video Generation via Mixture-of-Experts Latent Alignmentcited by 1
  11. 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolutioncited by 1
  12. 2026Physics-Grounded Fluid Video Generation with a Simulation Dataset and Dual-Stream Optical-Flow Supervisioncited by 1
11 more
  1. 2026Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decompositioncited by 1
  2. 2025Disentangled Geometric Alignment with Adaptive Contrastive Perturbation for Reliable Domain Transfercited by 0
  3. 2025Dynamic Epsilon Scheduling: A Multi-Factor Adaptive Perturbation Budget for Adversarial Trainingcited by 0
  4. 2026ReconMIL: Synergizing Latent Space Reconstruction with Bi-Stream Mamba for Whole Slide Image Analysiscited by 0
  5. 20263DPhysVideo: Consistency-Guided Flow SDE for Video Generation via 3D Scene Reconstruction and Physical Simulationcited by 0
  6. 2026Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Spacecited by 0
  7. 2026Vision Meets WiFi: Physics-Grounded Estimation of Volumetric Mechanical Propertiescited by 0
  8. 2026Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videoscited by 0
  9. 2026PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generationcited by 0
  10. 2026PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Imagecited by 0
  11. 2026NeuIDO: Neural Intrinsic Dynamics Operator for Physics-Informed 4D World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PhysX-3D: Physical-Grounded 3D Asset Generation
  2. 2024Precise-Physics Driven Text-to-3D Generation
  3. 2026PhysForge: Material-Controllable 3D Reconstruction with Separable Components
  4. 2026PhysReal: Learning Real-World Deformable Object Physics via Hybrid Constitutive Modeling
  5. 2026PhysMAS: Physics-Grounded Multi-Agent Synthesis of Compositional 4D Gaussians
  6. 2024Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials
  7. 2024MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets
  8. 2025MatPedia: A Universal Generative Foundation for High-Fidelity Material Synthesis