# PhiP-G: Physics-Guided Text-to-3D Compositional Scene Generation

arXiv: https://arxiv.org/abs/2502.00708 · HTML: https://mlsanity.com/paper/2502.00708/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 2 Feb 2025.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)
- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [Text-to-3D using Gaussian Splatting](https://mlsanity.com/paper/2309.16585/) (2023)
- [Holodeck: Language Guided Generation of 3D Embodied AI Environments](https://mlsanity.com/paper/2312.09067/) (2023)
- [LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes](https://mlsanity.com/paper/2311.13384/) (2023)
- [HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance](https://mlsanity.com/paper/2305.18766/) (2023)
- [GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting](https://mlsanity.com/paper/2402.07207/) (2024)
- [Compositional 3D Scene Generation using Locally Conditioned Diffusion](https://mlsanity.com/paper/2303.12218/) (2023)
- [Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenes](https://mlsanity.com/paper/2303.13450/) (2023)
- [CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layout](https://mlsanity.com/paper/2303.13843/) (2023)
- [NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis](https://mlsanity.com/paper/2003.08934/) (2020)
- [FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy](https://mlsanity.com/paper/2207.01390/) (2022)
- [AttnGrounder: Talking to Cars with Attention](https://mlsanity.com/paper/2009.05684/) (2020)

## Built on by (most cited first, 4 of 4)

- [PAT3D: Physics-Augmented Text-to-3D Scene Generation](https://mlsanity.com/paper/2511.21978/) (2025)
- [Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC](https://mlsanity.com/paper/2502.07007/) (2025)
- [Towards Geometric and Textural Consistency 3D Scene Generation via Single Image-guided Model Generation and Layout Optimization](https://mlsanity.com/paper/2507.14841/) (2025)
- [SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image](https://mlsanity.com/paper/2606.03994/) (2026)

## Abstract (arXiv)

Text-to-3D asset generation has achieved significant optimization under the supervision of 2D diffusion priors. However, when dealing with compositional scenes, existing methods encounter several challenges: 1). failure to ensure that composite scene layouts comply with physical laws; 2). difficulty in accurately capturing the assets and relationships described in complex scene descriptions; 3). limited autonomous asset generation capabilities among layout approaches leveraging large language models (LLMs). To avoid these compromises, we propose a novel framework for compositional scene generation, PhiP-G, which seamlessly integrates generation techniques with layout guidance based on a world model. Leveraging LLM-based agents, PhiP-G analyzes the complex scene description to generate a scene graph, and integrating a multimodal 2D generation agent and a 3D Gaussian generation method for targeted assets creation. For the stage of layout, PhiP-G employs a physical pool with adhesion capabilities and a visual supervision agent, forming a world model for layout prediction and planning. Extensive experiments demonstrate that PhiP-G significantly enhances the generation quality and physical rationality of the compositional scenes. Notably, PhiP-G attains state-of-the-art (SOTA) performance in CLIP scores, achieves parity with the leading methods in generation quality as measured by the T3Bench, and improves efficiency by 24x.