mlsanityData to 7 Oct 2026

ART3D: 3D Gaussian Splatting for Text-Guided Artistic Scenes Generation

arXiv 2405.10508 ↗First posted 17 May 2024cs.CVcited by 9

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

In this paper, we explore the existing challenges in 3D artistic scene generation by introducing ART3D, a novel framework that combines diffusion models and 3D Gaussian splatting techniques. Our method effectively bridges the gap between artistic and realistic images through an innovative image semantic transfer algorithm. By leveraging depth information and an initial artistic image, we generate a point cloud map, addressing domain differences. Additionally, we propose a depth consistency module to enhance 3D scene consistency. Finally, the 3D scene serves as initial points for optimizing Gaussian splats. Experimental results demonstrate ART3D's superior performance in both content and structural consistency metrics when compared to existing methods. ART3D significantly advances the field of AI in art creation by providing an innovative solution for generating high-quality 3D artistic scenes.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 43 of its 54 references that are in the map. First posted on arXiv 17 May 2024.

Builds on

43 of 54 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  2. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  3. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  4. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  5. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  6. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  7. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan
  8. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  8. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  9. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  10. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  11. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  12. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
23 more
  1. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  2. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  3. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  4. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  5. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  6. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  7. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  8. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 324
  9. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  10. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  11. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  12. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  13. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  14. 2020Consistent Video Depth Estimationcited by 220
  15. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  16. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  17. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  18. 2023LayerDiffusion: Layered Controlled Image Editing with Diffusion Modelscited by 40
  19. 2024Tuning-Free Image Customization with Image and Text Guidancecited by 25
  20. 2023Text-Guided Synthesis of Eulerian Cinemagraphscited by 22
  21. 2024Sketch-to-Architecture: Generative AI-aided Architectural Designcited by 4
  22. 2023Generative AI Model for Artistic Style Transfer Using Convolutional Neural Networkscited by 2
  23. 2024Generating Daylight-driven Architectural Design via Diffusion Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 1.

9 papers.

  1. 2024DepthLab: From Partial to Completecited by 27
  2. 2024SceneDreamer360: Text-Driven 3D-Consistent Scene Generation with Panoramic Gaussian Splattingcited by 24
  3. 2025FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placementcited by 24
  4. 2024HybridGS: Decoupling Transients and Statics with 2D and 3D Gaussian Splattingcited by 16
  5. 2025StyleMe3D: Stylization with Disentangled Priors by Multiple Encoders on 3D Gaussianscited by 6
  6. 2025EGGS: Exchangeable 2D/3D Gaussian Splatting for Geometry-Appearance Balanced Novel View Synthesiscited by 2
  7. 2024Challenges and Opportunities in 3D Content Generationcited by 1
  8. 2025From Fields to Splats: A Cross-Domain Survey of Real-Time Neural Scene Representationscited by 1
  9. 2026RoamScene3D: Immersive Text-to-3D Scene Generation via Adaptive Object-aware Roamingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024DreamScape: 3D Scene Creation via Gaussian Splatting joint Correlation Modeling
  2. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation
  3. 2024InstantStyleGaussian: Efficient Art Style Transfer with 3D Gaussian Splatting
  4. 2023CG3D: Compositional Generation for Text-to-3D via Gaussian Splatting
  5. 2025Generative Gaussian Splatting: Generating 3D Scenes with Video Diffusion Priors
  6. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting
  7. 2025GaussVideoDreamer: 3D Scene Generation with Video Diffusion and Inconsistency-Aware Gaussian Splatting
  8. 2026WINGS: Reference-Free Gaussian Splatting Inpainting with 3D-Native Generative Priors