mlsanityData to 7 Oct 2026

CG3D: Compositional Generation for Text-to-3D via Gaussian Splatting

arXiv 2311.17907 ↗First posted 29 Nov 2023cs.CV cs.AIcited by 42

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

With the onset of diffusion-based generative models and their ability to generate text-conditioned images, content generation has received a massive invigoration. Recently, these models have been shown to provide useful guidance for the generation of 3D graphics assets. However, existing work in text-conditioned 3D generation faces fundamental constraints: (i) inability to generate detailed, multi-object scenes, (ii) inability to textually control multi-object configurations, and (iii) physically realistic scene composition. In this work, we propose CG3D, a method for compositionally generating scalable 3D assets that resolves these constraints. We find that explicit Gaussian radiance fields, parameterized to allow for compositions of objects, possess the capability to enable semantically and physically consistent scenes. By utilizing a guidance framework built around this explicit representation, we show state of the art results, capable of even exceeding the guiding diffusion model in terms of object combinations and physics accuracy.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 45 of its 60 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

45 of 60 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts18 Oct
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  3. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  4. 2023CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layout24 Mar
  5. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation14 Mar
  6. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec
  7. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  9. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 36

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  9. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  10. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  12. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
24 more
  1. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  4. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  5. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  6. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  7. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  8. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 569
  9. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  10. 2022Blended Latent Diffusioncited by 430
  11. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  12. 2023Text-to-3D using Gaussian Splattingcited by 299
  13. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  14. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  15. 2021Learning Object-Compositional Neural Radiance Field for Editable Scene Renderingcited by 228
  16. 2022SINE: SINgle Image Editing with Text-to-Image Diffusion Modelscited by 159
  17. 2021BACON: Band-limited Coordinate Networks for Multiscale Scene Representationcited by 142
  18. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  19. 2023Score-Based Diffusion Models as Principled Priors for Inverse Imagingcited by 126
  20. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  21. 2018Human-centric Indoor Scene Synthesis Using Stochastic Grammarcited by 73
  22. 2021Scene Synthesis via Uncertainty-Driven Attribute Synchronizationcited by 29
  23. 2021Indoor Scene Generation from a Collection of Semantic-Segmented Depth Imagescited by 27
  24. 2022ORCa: Glossy Objects as Radiance Field Camerascited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 27; 2025: 14; 2026: 1.

42 papers.

Sort:

By area:

  1. 2024A Survey on 3D Gaussian Splattingcited by 221
  2. 2024Recent Advances in 3D Gaussian Splattingcited by 130
  3. 2024GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splattingcited by 104
  4. 20243D Gaussian as a New Era: A Surveycited by 102
  5. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
  6. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 77
  7. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  8. 2024DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Samplingcited by 72
  9. 2024ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidancecited by 59
  10. 20244K4DGen: Panoramic 4D Generation at 4K Resolutioncited by 54
  11. 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
  12. 2024Comp4D: LLM-Guided Compositional 4D Scene Generationcited by 50
30 more
  1. 2024REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignmentcited by 45
  2. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  3. 2024Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Reviewcited by 40
  4. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  5. 2024ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillationcited by 27
  6. 2025Generating Physically Stable and Buildable Brick Structures from Textcited by 23
  7. 2024Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprintcited by 21
  8. 2024Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabricationcited by 19
  9. 2025A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generationcited by 19
  10. 2025LL3M: Large Language 3D Modelerscited by 18
  11. 2024EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editingcited by 14
  12. 2025PAT3D: Physics-Augmented Text-to-3D Scene Generationcited by 14
  13. 2024Text-to-3D Shape Generationcited by 13
  14. 2024Efficient Density Control for 3D Gaussian Splattingcited by 12
  15. 2025VideoRFSplat: Direct Scene-Level Text-to-3D Gaussian Splatting Generation with Flexible Pose and Multi-View Joint Modelingcited by 12
  16. 2025Compass Control: Multi Object Orientation Control for Text-to-Image Generationcited by 11
  17. 2024DreamHOI: Subject-Driven Generation of 3D Human-Object Interactions with Diffusion Priorscited by 9
  18. 2024Semantic Score Distillation Sampling for Compositional Text-to-3D Generationcited by 9
  19. 2024DreamScape: 3D Scene Creation via Gaussian Splatting joint Correlation Modelingcited by 7
  20. 2025Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Modelscited by 7
  21. 2024Scene123: One Prompt to 3D Scene Generation via Video-Assisted and Consistency-Enhanced MAEcited by 6
  22. 2025SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesiscited by 6
  23. 2024DreamBeast: Distilling 3D Fantastical Animals with Part-Aware Knowledge Transfercited by 5
  24. 2025Self-Evolving 3D Scene Generation from a Single Imagecited by 3
  25. 2025Constructing a 3D Scene from a Single Imagecited by 2
  26. 2025Part2GS: Part-aware Modeling of Articulated Objects using 3D Gaussian Splattingcited by 1
  27. 2025DreamScene: 3D Gaussian-based End-to-end Text-to-3D Scene Generationcited by 1
  28. 2025From Fields to Splats: A Cross-Domain Survey of Real-Time Neural Scene Representationscited by 1
  29. 2026Co-generation of Layout and Shape from Text via Autoregressive 3D Diffusioncited by 1
  30. 2025Drag4D: Align Your Motion with Text-Driven 3D Scene Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SIC3D: Style Image Conditioned Text-to-3D Gaussian Splatting Generation
  2. 2025Generative Gaussian Splatting: Generating 3D Scenes with Video Diffusion Priors
  3. 2026PixGS: Pixel-Space Diffusion for Direct 3D Gaussian Splat Generation
  4. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation
  5. 2024ART3D: 3D Gaussian Splatting for Text-Guided Artistic Scenes Generation
  6. 2025Generating Surface for Text-to-3D using 2D Gaussian Splatting
  7. 2024Enhancing Single Image to 3D Generation using Gaussian Splatting and Hybrid Diffusion Priors
  8. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models