mlsanityData to 7 Oct 2026

GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models

arXiv 2310.08529 ↗First posted 12 Oct 2023cs.CV cs.GRcited by 255

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

In recent times, the generation of 3D assets from text prompts has shown impressive results. Both 2D and 3D diffusion models can help generate decent 3D objects based on prompts. 3D diffusion models have good 3D consistency, but their quality and generalization are limited as trainable 3D data is expensive and hard to obtain. 2D diffusion models enjoy strong abilities of generalization and fine generation, but 3D consistency is hard to guarantee. This paper attempts to bridge the power from the two types of diffusion models via the recent explicit and efficient 3D Gaussian splatting representation. A fast 3D object generation framework, named as GaussianDreamer, is proposed, where the 3D diffusion model provides priors for initialization and the 2D diffusion model enriches the geometry and appearance. Operations of noisy point growing and color perturbation are introduced to enhance the initialized Gaussians. Our GaussianDreamer can generate a high-quality 3D instance or 3D avatar within 15 minutes on one GPU, much faster than previous methods, while the generated instances can be directly rendered in real time. Demos and code are available at https://taoranyi.com/gaussiandreamer/.

Key facts. Cited by 255 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 78 of its 102 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

78 of 102 references.

Closest ancestors · 33

Same direction, newest first

  1. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models4 Oct
  2. 2023T3Bench: Benchmarking Current Progress in Text-to-3D Generation4 Oct
  3. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  4. 2023Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priors29 Sep
  5. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  6. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  7. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis22 Aug
  8. 2023HumanLiff: Layer-wise 3D Human Generation with Diffusion Model18 Aug
  9. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humans16 Aug
  10. 2023HD-Fusion: Detailed Text-to-3D Generation Leveraging Multiple Noise Estimation30 Jul
  11. 2023Points-to-3D: Bridging the Gap between Sparse Points and Shape-Controllable Text-to-3D Generation26 Jul
  12. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
21 more
  1. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  2. 2023HeadSculpt: Crafting 3D Head Avatars with Text5 Jun
  3. 2023ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image25 May
  4. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  5. 2023RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture18 May
  6. 2023TextMesh: Generation of Realistic 3D Meshes From Text Prompts24 Apr
  7. 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models3 Apr
  8. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  9. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  10. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  11. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation14 Mar
  12. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generation9 Mar
  13. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec
  14. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  15. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  16. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  17. 20223DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models25 Nov
  18. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  19. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  20. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  21. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  6. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  7. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  9. 2022Human Motion Diffusion Modelcited by 1,062
  10. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  11. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  12. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
33 more
  1. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  2. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  3. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  4. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  5. 2023Human Motion Diffusion as a Generative Priorcited by 354
  6. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  7. 2023Text-to-3D using Gaussian Splattingcited by 299
  8. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 282
  9. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  10. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  11. 2019Differentiable Surface Splatting for Point-based Geometry Processingcited by 238
  12. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  13. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  14. 2022FLAME: Free-form Language-based Motion Synthesis & Editingcited by 217
  15. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  16. 2023MeshDiffusion: Score-based Generative 3D Mesh Modelingcited by 198
  17. 2021Point-Based Neural Rendering with Per-View Optimizationcited by 188
  18. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  19. 2023GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latentscited by 156
  20. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  21. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  22. 2023AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Controlcited by 84
  23. 2022TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decompositioncited by 83
  24. 2023DreamWaltz: Make a Scene with Complex 3D Animatable Avatarscited by 82
  25. 2023Neural Point Catacaustics for Novel-View Synthesis of Reflectionscited by 81
  26. 2023DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidancecited by 79
  27. 2022UDE: A Unified Driving Engine for Human Motion Generationcited by 67
  28. 2023Single Motion Diffusioncited by 65
  29. 2023AvatarVerse: High-quality & Stable 3D Avatar Creation from Text and Posecited by 60
  30. 2023DITTO-NeRF: Diffusion-based Iterative Text To Omni-directional 3D Modelcited by 43
  31. 2023Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Modelscited by 17
  32. 2023VPP: Efficient Conditional 3D Generation via Voxel-Point Progressive Representationcited by 14
  33. 2023Chasing Consistency in Text-to-3D Generation from a Single Imagecited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 84; 2025: 107; 2026: 62.

255 papers.

Sort:

By area:

  1. 2024A Survey on 3D Gaussian Splattingcited by 221
  2. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  3. 2024Recent Advances in 3D Gaussian Splattingcited by 130
  4. 2024RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPScited by 110
  5. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  6. 20243DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priorscited by 106
  7. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 77
  8. 2024Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splattingcited by 73
  9. 2024End-to-End Rate-Distortion Optimized 3D Gaussian Representationcited by 71
  10. 2024R2-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstructioncited by 65
  11. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  12. 2024VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Modelscited by 56
38 more
  1. 2024LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Modelscited by 51
  2. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
  3. 2024EgoLifter: Open-world 3D Segmentation for Egocentric Perceptioncited by 46
  4. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  5. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  6. 2024REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignmentcited by 45
  7. 2025SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generationcited by 44
  8. 2024Instant3dit: Multiview Inpainting for Fast Editing of 3D Objectscited by 43
  9. 2025Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attentioncited by 43
  10. 2024L3DG: Latent 3D Gaussian Diffusioncited by 42
  11. 2024Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Reviewcited by 40
  12. 2025Deformable Beta Splattingcited by 38
  13. 2025UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Modelscited by 38
  14. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  15. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  16. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 34
  17. 2024Relaxing Accurate Initialization Constraint for 3D Gaussian Splattingcited by 33
  18. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 33
  19. 2024SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decompositioncited by 29
  20. 2024Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectivescited by 27
  21. 2024PrEditor3D: Fast and Precise 3D Shape Editingcited by 27
  22. 2024GS-Hider: Hiding Messages into 3D Gaussian Splattingcited by 26
  23. 2025CityGS-X: A Scalable Architecture for Efficient and Geometrically Accurate Large-Scale Scene Reconstructioncited by 26
  24. 2025FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placementcited by 24
  25. 2024ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretrainingcited by 21
  26. 2024Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprintcited by 21
  27. 2025Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuningcited by 21
  28. 20243DSceneEditor: Controllable 3D Scene Editing with Gaussian Splattingcited by 20
  29. 2024Hash3D: Training-free Acceleration for 3D Generationcited by 19
  30. 2024GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Detailscited by 19
  31. 2025A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generationcited by 19
  32. 2024Mirror-3DGS: Incorporating Mirror Reflections into 3D Gaussian Splattingcited by 18
  33. 2024GuardSplat: Efficient and Robust Watermarking for 3D Gaussian Splattingcited by 18
  34. 2025HiScene: Creating Hierarchical 3D Scenes with Isometric View Generationcited by 18
  35. 2025LL3M: Large Language 3D Modelerscited by 18
  36. 2024Poison-splat: Computation Cost Attack on 3D Gaussian Splattingcited by 17
  37. 2025PanoGS: Gaussian-based Panoptic Segmentation for 3D Open Vocabulary Scene Understandingcited by 16
  38. 2025Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagerycited by 16

50 most cited of 255 All 255 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Repurposing 2D Diffusion Models with Gaussian Atlas for 3D Generation
  2. 2023EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior
  3. 2024A General Framework to Boost 3D GS Initialization for Text-to-3D Generation by Lexical Richness
  4. 2024DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data
  5. 2023PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion
  6. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation
  7. 2024Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representation
  8. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors