# PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Image

arXiv: https://arxiv.org/abs/2411.18548 · HTML: https://mlsanity.com/paper/2411.18548/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 55 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model](https://mlsanity.com/paper/2310.15110/) (2023)
- [CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets](https://mlsanity.com/paper/2406.13897/) (2024)
- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)
- [Locally Attentional SDF Diffusion for Controllable 3D Shape Generation](https://mlsanity.com/paper/2305.04461/) (2023)
- [Compositional 3D Scene Generation using Locally Conditioned Diffusion](https://mlsanity.com/paper/2303.12218/) (2023)
- [SAMPart3D: Segment Any Part in 3D Objects](https://mlsanity.com/paper/2411.07184/) (2024)
- [Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion](https://mlsanity.com/paper/2406.04338/) (2024)
- [ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidance](https://mlsanity.com/paper/2403.12409/) (2024)
- [SceneWiz3D: Towards Text-guided 3D Scene Composition](https://mlsanity.com/paper/2312.08885/) (2023)
- [Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing](https://mlsanity.com/paper/2404.01223/) (2024)
- [PhyRecon: Physically Plausible Neural Scene Reconstruction](https://mlsanity.com/paper/2404.16666/) (2024)
- [Learning Disentangled Avatars with Hybrid 3D Representations](https://mlsanity.com/paper/2309.06441/) (2023)
- [Physically Compatible 3D Object Modeling from a Single Image](https://mlsanity.com/paper/2405.20510/) (2024)
- [Disentangled Clothed Avatar Generation from Text Descriptions](https://mlsanity.com/paper/2312.05295/) (2023)
- [Disentangled 3D Scene Generation with Layout Learning](https://mlsanity.com/paper/2402.16936/) (2024)
- [Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Plane](https://mlsanity.com/paper/2403.16210/) (2024)
- [HumanLiff: Layer-wise 3D Human Generation with Diffusion Model](https://mlsanity.com/paper/2308.09712/) (2023)
- [TELA: Text to Layer-wise 3D Clothed Human Generation](https://mlsanity.com/paper/2404.16748/) (2024)

## Built on by (most cited first, 19 of 19)

- [OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesion](https://mlsanity.com/paper/2507.06165/) (2025)
- [X-Part: high fidelity and structure coherent shape decomposition](https://mlsanity.com/paper/2509.08643/) (2025)
- [Generative Physical AI in Vision: A Survey](https://mlsanity.com/paper/2501.10928/) (2025)
- [MV-SAM3D: Adaptive Multi-View Fusion for Layout-Aware 3D Generation](https://mlsanity.com/paper/2603.11633/) (2026)
- [PAT3D: Physics-Augmented Text-to-3D Scene Generation](https://mlsanity.com/paper/2511.21978/) (2025)
- [Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC](https://mlsanity.com/paper/2502.07007/) (2025)
- [PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual World](https://mlsanity.com/paper/2605.05163/) (2026)
- [Part-X-MLLM: Part-aware 3D Multimodal Large Language Model](https://mlsanity.com/paper/2511.13647/) (2025)
- [Is Single-View Mesh Reconstruction Ready for Robotics?](https://mlsanity.com/paper/2505.17966/) (2025)
- [SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image](https://mlsanity.com/paper/2606.03994/) (2026)
- [PartDiffuser: Part-wise 3D Mesh Generation via Discrete Diffusion](https://mlsanity.com/paper/2511.18801/) (2025)
- [CADKnitter: Compositional CAD Generation from Text and Geometry Guidance](https://mlsanity.com/paper/2512.11199/) (2025)
- [HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactions](https://mlsanity.com/paper/2603.15612/) (2026)
- [DM-CFO: A Diffusion Model for Compositional 3D Tooth Generation with Collision-Free Optimization](https://mlsanity.com/paper/2603.03602/) (2026)
- [3D Generation for Embodied AI and Robotic Simulation: A Survey](https://mlsanity.com/paper/2604.26509/) (2026)
- [PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assets](https://mlsanity.com/paper/2608.08053/) (2026)
- [Muses: Designing, Composing, Generating Nonexistent Fantasy 3D Creatures without Training](https://mlsanity.com/paper/2601.03256/) (2026)
- [Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editing](https://mlsanity.com/paper/2608.02711/) (2026)
- [KaiNinja: Extending Native 3D Generators to the Part Level](https://mlsanity.com/paper/2609.15659/) (2026)

## Abstract (arXiv)

We present PhyCAGE, the first approach for physically plausible compositional 3D asset generation from a single image. Given an input image, we first generate consistent multi-view images for components of the assets. These images are then fitted with 3D Gaussian Splatting representations. To ensure that the Gaussians representing objects are physically compatible with each other, we introduce a Physical Simulation-Enhanced Score Distillation Sampling (PSE-SDS) technique to further optimize the positions of the Gaussians. It is achieved by setting the gradient of the SDS loss as the initial velocity of the physical simulation, allowing the simulator to act as a physics-guided optimizer that progressively corrects the Gaussians' positions to a physically compatible state. Experimental results demonstrate that the proposed method can generate physically plausible compositional 3D assets given a single image.