mlsanityData to 14 Oct 2026

Repurposing 2D Diffusion Models with Gaussian Atlas for 3D Generation

arXiv 2503.15877 ↗First posted 20 Mar 2025cs.CVcited by 7

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Recent advances in text-to-image diffusion models have been driven by the increasing availability of paired 2D data. However, the development of 3D diffusion models has been hindered by the scarcity of high-quality 3D data, resulting in less competitive performance compared to their 2D counterparts. To address this challenge, we propose repurposing pre-trained 2D diffusion models for 3D object generation. We introduce Gaussian Atlas, a novel representation that utilizes dense 2D grids, enabling the fine-tuning of 2D diffusion models to generate 3D Gaussians. Our approach demonstrates successful transfer learning from a pre-trained 2D diffusion model to a 2D manifold flattened from 3D structures. To support model training, we compile GaussianVerse, a large-scale dataset comprising 205K high-quality 3D Gaussian fittings of various 3D objects. Our experimental results show that text-to-image diffusion models can be effectively adapted for 3D content generation, bridging the gap between 2D and 3D modeling.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 51 of its 64 references that are in the map. First posted on arXiv 20 Mar 2025.

Builds on

51 of 64 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation28 Jan
  2. 2024DiffGS: Functional Gaussian Splatting Diffusion25 Oct
  3. 2024GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction5 Jul
  4. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image18 Mar
  5. 2023PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion14 Dec
  6. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
  7. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  8. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  9. 20223D Neural Field Generation using Triplane Diffusion30 Nov
  10. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  11. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 40

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  10. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  11. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  12. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
28 more
  1. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  2. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  3. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  4. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  5. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 695
  6. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  7. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  8. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  9. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
  10. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 364
  11. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  12. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
  13. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 232
  14. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 215
  15. 2023Scalable 3D Captioning with Pretrained Modelscited by 203
  16. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 189
  17. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 101
  18. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 92
  19. 2024What Matters When Repurposing Diffusion Models for General Dense Perception Tasks?cited by 74
  20. 2024GVGEN: Text-to-3D Generation with Volumetric Representationcited by 51
  21. 2024Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Setcited by 49
  22. 2024L3DG: Latent 3D Gaussian Diffusioncited by 42
  23. 2022FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstructioncited by 39
  24. 2024An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusioncited by 34
  25. 2024HexaGen3D: StableDiffusion is just one step away from Fast and Diverse Text-to-3D Generationcited by 21
  26. 2024ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretrainingcited by 21
  27. 2024Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representationcited by 17
  28. 2024Flatten Anything: Unsupervised Neural Surface Parameterizationcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025TRIM: Scalable 3D Gaussian Diffusion Inference with Temporal and Spatial Trimmingcited by 3
  2. 2025Complete Gaussian Splats from a Single Image with Denoising Diffusion Modelscited by 2
  3. 2026PackUV: Packed Gaussian UV Maps for 4D Volumetric Videocited by 2
  4. 2025Artist-Created Mesh Generation from Raw Observationcited by 1
  5. 2025Repurposing 2D Diffusion Models for 3D Shape Completioncited by 1
  6. 2026Sparse auto-regressive modeling for scene generation from multi-view imagescited by 1
  7. 2026A Survey on Adversarial Attacks and Defenses for Diffusion Models Across Multiple Modalitiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Atlas Gaussians Diffusion for 3D Generation
  2. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models
  3. 2024Enhancing Single Image to 3D Generation using Gaussian Splatting and Hybrid Diffusion Priors
  4. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation
  5. 2025Zero-1-to-G: Taming Pretrained 2D Diffusion Model for Direct 3D Generation
  6. 2025Distilling Multi-view Diffusion Models into 3D Generators
  7. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models
  8. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Images