mlsanityData to 14 Oct 2026

PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Image

arXiv 2411.18548 ↗First posted 27 Nov 2024cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

We present PhyCAGE, the first approach for physically plausible compositional 3D asset generation from a single image. Given an input image, we first generate consistent multi-view images for components of the assets. These images are then fitted with 3D Gaussian Splatting representations. To ensure that the Gaussians representing objects are physically compatible with each other, we introduce a Physical Simulation-Enhanced Score Distillation Sampling (PSE-SDS) technique to further optimize the positions of the Gaussians. It is achieved by setting the gradient of the SDS loss as the initial velocity of the physical simulation, allowing the simulator to act as a physics-guided optimizer that progressively corrects the Gaussians' positions to a physically compatible state. Experimental results demonstrate that the proposed method can generate physically plausible compositional 3D assets given a single image.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 32 of its 55 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

32 of 55 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2023Segment Anythingcited by 8,234
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  8. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  9. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  10. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  11. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  12. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
20 more
  1. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
  2. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
  3. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  4. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 130
  5. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  6. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 97
  7. 2024Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusioncited by 80
  8. 2024ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidancecited by 59
  9. 2023SceneWiz3D: Towards Text-guided 3D Scene Compositioncited by 54
  10. 2024Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editingcited by 45
  11. 2024PhyRecon: Physically Plausible Neural Scene Reconstructioncited by 45
  12. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  13. 2024Physically Compatible 3D Object Modeling from a Single Imagecited by 39
  14. 2023Disentangled Clothed Avatar Generation from Text Descriptionscited by 33
  15. 2024Disentangled 3D Scene Generation with Layout Learningcited by 33
  16. 2024Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Planecited by 26
  17. 2023HumanLiff: Layer-wise 3D Human Generation with Diffusion Modelcited by 23
  18. 2024TELA: Text to Layer-wise 3D Clothed Human Generationcited by 20
  19. 2024Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabricationcited by 20
  20. 2023Layered 3D Human Generation via Semantic-Aware Diffusion Modelcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 10.

19 papers.

Sort:

By area:

  1. 2025OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesioncited by 61
  2. 2025X-Part: high fidelity and structure coherent shape decompositioncited by 44
  3. 2025Generative Physical AI in Vision: A Surveycited by 24
  4. 2026MV-SAM3D: Adaptive Multi-View Fusion for Layout-Aware 3D Generationcited by 19
  5. 2025PAT3D: Physics-Augmented Text-to-3D Scene Generationcited by 16
  6. 2025Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGCcited by 12
  7. 2026PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual Worldcited by 9
  8. 2025Part-X-MLLM: Part-aware 3D Multimodal Large Language Modelcited by 7
  9. 2025Is Single-View Mesh Reconstruction Ready for Robotics?cited by 6
  10. 2026SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Imagecited by 5
  11. 2025PartDiffuser: Part-wise 3D Mesh Generation via Discrete Diffusioncited by 4
  12. 2025CADKnitter: Compositional CAD Generation from Text and Geometry Guidancecited by 2
7 more
  1. 2026HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactionscited by 2
  2. 2026DM-CFO: A Diffusion Model for Compositional 3D Tooth Generation with Collision-Free Optimizationcited by 1
  3. 20263D Generation for Embodied AI and Robotic Simulation: A Surveycited by 1
  4. 2026PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assetscited by 1
  5. 2026Muses: Designing, Composing, Generating Nonexistent Fantasy 3D Creatures without Trainingcited by 0
  6. 2026Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editingcited by 0
  7. 2026KaiNinja: Extending Native 3D Generators to the Part Levelcited by 0