# L3DG: Latent 3D Gaussian Diffusion

arXiv: https://arxiv.org/abs/2410.13530 · HTML: https://mlsanity.com/paper/2410.13530/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 52 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [Novel View Synthesis with Diffusion Models](https://mlsanity.com/paper/2210.04628/) (2022)
- [Text-to-3D using Gaussian Splatting](https://mlsanity.com/paper/2309.16585/) (2023)
- [GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models](https://mlsanity.com/paper/2310.08529/) (2023)
- [Learning Gradient Fields for Shape Generation](https://mlsanity.com/paper/2008.06520/) (2020)
- [DiffRF: Rendering-Guided 3D Radiance Field Diffusion](https://mlsanity.com/paper/2212.01206/) (2022)
- [Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs](https://mlsanity.com/paper/1703.09438/) (2017)
- [Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction](https://mlsanity.com/paper/2304.06714/) (2023)
- [VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids](https://mlsanity.com/paper/2206.07695/) (2022)
- [latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction](https://mlsanity.com/paper/2403.16292/) (2024)
- [EpiGRAF: Rethinking training of 3D GANs](https://mlsanity.com/paper/2206.10535/) (2022)

## Built on by (most cited first, 30 of 42)

- [MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation](https://mlsanity.com/paper/2412.03558/) (2024)
- [Bolt3D: Generating 3D Scenes in Seconds](https://mlsanity.com/paper/2503.14445/) (2025)
- [Amodal3R: Amodal 3D Reconstruction from Occluded 2D Images](https://mlsanity.com/paper/2503.13439/) (2025)
- [VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Space](https://mlsanity.com/paper/2508.19247/) (2025)
- [MeshArt: Generating Articulated Meshes with Structure-Guided Transformers](https://mlsanity.com/paper/2412.11596/) (2024)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation](https://mlsanity.com/paper/2412.21117/) (2024)
- [Gen3R: 3D Scene Generation Meets Feed-Forward Reconstruction](https://mlsanity.com/paper/2601.04090/) (2026)
- [OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generation](https://mlsanity.com/paper/2504.09975/) (2025)
- [HybridGS: Decoupling Transients and Statics with 2D and 3D Gaussian Splatting](https://mlsanity.com/paper/2412.03844/) (2024)
- [SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting](https://mlsanity.com/paper/2506.08710/) (2025)
- [QuickSplat: Fast 3D Surface Reconstruction via Learned Gaussian Initialization](https://mlsanity.com/paper/2505.05591/) (2025)
- [SceneTok: A Compressed, Diffusable Token Space for 3D Scenes](https://mlsanity.com/paper/2602.18882/) (2026)
- [SegviGen: Repurposing 3D Generative Model for Part Segmentation](https://mlsanity.com/paper/2603.16869/) (2026)
- [A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervision](https://mlsanity.com/paper/2412.00623/) (2024)
- [DNF: Unconditional 4D Generation with Dictionary-based Neural Fields](https://mlsanity.com/paper/2412.05161/) (2024)
- [WorldClaw: Agentic 3D Open-World Generation at Scale](https://mlsanity.com/paper/2608.05248/) (2026)
- [Repurposing 2D Diffusion Models with Gaussian Atlas for 3D Generation](https://mlsanity.com/paper/2503.15877/) (2025)
- [Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data](https://mlsanity.com/paper/2503.21694/) (2025)
- [SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledge](https://mlsanity.com/paper/2512.01629/) (2025)
- [Can3Tok: Canonical 3D Tokenization and Latent Modeling of Scene-Level 3D Gaussians](https://mlsanity.com/paper/2508.01464/) (2025)
- [GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation](https://mlsanity.com/paper/2603.26661/) (2026)
- [GenAssets: Generating in-the-wild 3D Assets in Latent Space](https://mlsanity.com/paper/2604.23010/) (2026)
- [GSV3D: Gaussian Splatting-based Geometric Distillation with Stable Video Diffusion for Single-Image 3D Object Generation](https://mlsanity.com/paper/2503.06136/) (2025)
- [Repurposing 3D Generative Model for Autoregressive Layout Generation](https://mlsanity.com/paper/2604.16299/) (2026)
- [Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction](https://mlsanity.com/paper/2604.04874/) (2026)
- [Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Data](https://mlsanity.com/paper/2604.13688/) (2026)
- [HuGDiffusion: Generalizable Single-Image Human Rendering via 3D Gaussian Diffusion](https://mlsanity.com/paper/2501.15008/) (2025)
- [Object-X: Learning to Reconstruct Multi-Modal 3D Object Representations](https://mlsanity.com/paper/2506.04789/) (2025)
- [Hyper Diffusion Avatars: Dynamic Human Avatar Generation using Network Weight Space Diffusion](https://mlsanity.com/paper/2509.04145/) (2025)

## Abstract (arXiv)

We propose L3DG, the first approach for generative 3D modeling of 3D Gaussians through a latent 3D Gaussian diffusion formulation. This enables effective generative 3D modeling, scaling to generation of entire room-scale scenes which can be very efficiently rendered. To enable effective synthesis of 3D Gaussians, we propose a latent diffusion formulation, operating in a compressed latent space of 3D Gaussians. This compressed latent space is learned by a vector-quantized variational autoencoder (VQ-VAE), for which we employ a sparse convolutional architecture to efficiently operate on room-scale scenes. This way, the complexity of the costly generation process via diffusion is substantially reduced, allowing higher detail on object-level generation, as well as scalability to large scenes. By leveraging the 3D Gaussian representation, the generated scenes can be rendered from arbitrary viewpoints in real-time. We demonstrate that our approach significantly improves visual quality over prior work on unconditional object-level radiance field synthesis and showcase its applicability to room-scale scene generation.