mlsanityData to 7 Oct 2026

Atlas Gaussians Diffusion for 3D Generation

arXiv 2408.13055 ↗First posted 23 Aug 2024cs.CVcited by 15

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Using the latent diffusion model has proven effective in developing novel 3D generation techniques. To harness the latent diffusion model, a key challenge is designing a high-fidelity and efficient representation that links the latent space and the 3D space. In this paper, we introduce Atlas Gaussians, a novel representation for feed-forward native 3D generation. Atlas Gaussians represent a shape as the union of local patches, and each patch can decode 3D Gaussians. We parameterize a patch as a sequence of feature vectors and design a learnable function to decode 3D Gaussians from the feature vectors. In this process, we incorporate UV-based sampling, enabling the generation of a sufficiently large, and theoretically infinite, number of 3D Gaussian points. The large amount of 3D Gaussians enables the generation of high-quality details. Moreover, due to local awareness of the representation, the transformer-based decoding procedure operates on a patch level, ensuring efficiency. We train a variational autoencoder to learn the Atlas Gaussians representation, and then apply a latent diffusion model on its latent space for learning 3D Generation. Experiments show that our approach outperforms the prior arts of feed-forward native 3D generation. Project page: https://yanghtr.github.io/projects/atlas_gaussians.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 106 of its 124 references that are in the map. First posted on arXiv 23 Aug 2024.

Builds on

106 of 124 references.

Closest ancestors · 27

Same direction, newest first

  1. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets30 May
  2. 2024VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models18 Mar
  3. 2024V3D: Video Diffusion Models are Effective 3D Generators11 Mar
  4. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation2 Dec
  5. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D28 Nov
  6. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  7. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
  8. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior25 Oct
  9. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  10. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  11. 2023Large-Vocabulary 3D Diffusion Model with Transformer14 Sep
  12. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
15 more
  1. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  2. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  3. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  4. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction13 Apr
  5. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  6. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models26 Jan
  7. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  8. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  9. 20223D Neural Field Generation using Triplane Diffusion30 Nov
  10. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  11. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation17 Nov
  12. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  13. 2022Novel View Synthesis with Diffusion Models6 Oct
  14. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  15. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 79

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
38 more
  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  11. 2022Video Diffusion Modelscited by 2,280
  12. 2015Variational Inference with Normalizing Flowscited by 2,161
  13. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  14. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  15. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  16. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  17. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  18. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  19. 2022Human Motion Diffusion Modelcited by 1,062
  20. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
  21. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  22. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  23. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  24. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  25. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  26. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  27. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  28. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  29. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 611
  30. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  31. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  32. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  33. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  34. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  35. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  36. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  37. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  38. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 5.

15 papers.

Sort:

By area:

  1. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  2. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  3. 2025Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesiscited by 33
  4. 2025UniLat3D: Geometry-Appearance Unified Latents for Single-Stage 3D Generationcited by 20
  5. 2025A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generationcited by 19
  6. 2024GaussianAnything: Interactive Point Cloud Flow Matching For 3D Object Generationcited by 12
  7. 2025SIGMAN:Scaling 3D Human Gaussian Generation with Millions of Assetscited by 12
  8. 2026LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windowscited by 2
  9. 2026WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenescited by 2
  10. 2025Artist-Created Mesh Generation from Raw Observationcited by 1
  11. 2025Repurposing 2D Diffusion Models for 3D Shape Completioncited by 1
  12. 2026Text-Image Conditioned 3D Generationcited by 1
3 more
  1. 2025InStyle: Instant Appearance Stylization of 3D Shapescited by 0
  2. 2026ARGS: Auto-Regressive Gaussian Splatting via Parallel Progressive Next-Scale Predictioncited by 0
  3. 2026Visibility-Guided Structured Measure Flow for Class-Conditioned 3D Gaussian Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Repurposing 2D Diffusion Models with Gaussian Atlas for 3D Generation
  2. 2024L3DG: Latent 3D Gaussian Diffusion
  3. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models
  4. 2026Generative 3D Gaussians with Learned Density Control
  5. 2025Distilling Multi-view Diffusion Models into 3D Generators
  6. 2024Enhancing Single Image to 3D Generation using Gaussian Splatting and Hybrid Diffusion Priors
  7. 2024Large Point-to-Gaussian Model for Image-to-3D Generation
  8. 2026FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation