# GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling

arXiv: https://arxiv.org/abs/2403.19655 · HTML: https://mlsanity.com/paper/2403.19655/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 79 references that are in the map.
- First posted on arXiv 28 Mar 2024.

## Builds on (most cited first, 30 of 63)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)

## Built on by (most cited first, 30 of 45)

- [PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers](https://mlsanity.com/paper/2506.05573/) (2025)
- [SparseFlex: High-Resolution and Arbitrary-Topology 3D Shape Modeling](https://mlsanity.com/paper/2503.21732/) (2025)
- [SceneGen: Single-Image 3D Scene Generation in One Feedforward Pass](https://mlsanity.com/paper/2508.15769/) (2025)
- [DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation](https://mlsanity.com/paper/2501.16764/) (2025)
- [Improved Noise Schedule for Diffusion Training](https://mlsanity.com/paper/2407.03297/) (2024)
- [Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesis](https://mlsanity.com/paper/2507.23785/) (2025)
- [VideoRFSplat: Direct Scene-Level Text-to-3D Gaussian Splatting Generation with Flexible Pose and Multi-View Joint Modeling](https://mlsanity.com/paper/2503.15855/) (2025)
- [DetailGen3D: Generative 3D Geometry Enhancement via Data-Dependent Flow](https://mlsanity.com/paper/2411.16820/) (2024)
- [Graph Canvas for Controllable 3D Scene Generation](https://mlsanity.com/paper/2412.00091/) (2024)
- [Learnable Infinite Taylor Gaussian for Dynamic View Rendering](https://mlsanity.com/paper/2412.04282/) (2024)
- [Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generator](https://mlsanity.com/paper/2510.13454/) (2025)
- [Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data](https://mlsanity.com/paper/2503.21694/) (2025)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [FRIDU: Functional Map Refinement with Guided Image Diffusion](https://mlsanity.com/paper/2506.14322/) (2025)
- [MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphing](https://mlsanity.com/paper/2601.00204/) (2026)
- [Generative 3D Gaussians with Learned Density Control](https://mlsanity.com/paper/2605.16355/) (2026)
- [Event-boosted Deformable 3D Gaussians for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2411.16180/) (2024)
- [VRsketch2Gaussian: 3D VR Sketch Guided 3D Object Generation with Gaussian Splatting](https://mlsanity.com/paper/2503.12383/) (2025)
- [Sculpt4D: Generating 4D Shapes via Sparse-Attention Diffusion Transformers](https://mlsanity.com/paper/2604.21592/) (2026)
- [LoG3D: Ultra-High-Resolution 3D Shape Modeling via Local-to-Global Partitioning](https://mlsanity.com/paper/2511.10040/) (2025)
- [Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction](https://mlsanity.com/paper/2604.04874/) (2026)
- [PixGS: Pixel-Space Diffusion for Direct 3D Gaussian Splat Generation](https://mlsanity.com/paper/2607.01803/) (2026)
- [GS-Voxel: Fitting-Free Structured Latents for Large-Scale 3DGS Generation](https://mlsanity.com/paper/2608.17988/) (2026)
- [HuGDiffusion: Generalizable Single-Image Human Rendering via 3D Gaussian Diffusion](https://mlsanity.com/paper/2501.15008/) (2025)
- [UniView: Enhancing Novel View Synthesis From A Single Image By Unifying Reference Features](https://mlsanity.com/paper/2509.04932/) (2025)
- [Artist-Created Mesh Generation from Raw Observation](https://mlsanity.com/paper/2509.12501/) (2025)
- [Inferring Compositional 4D Scenes without Ever Seeing One](https://mlsanity.com/paper/2512.05272/) (2025)
- [Repurposing 2D Diffusion Models for 3D Shape Completion](https://mlsanity.com/paper/2512.13991/) (2025)
- [GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos](https://mlsanity.com/paper/2604.07273/) (2026)
- [Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes](https://mlsanity.com/paper/2604.14914/) (2026)

## Abstract (arXiv)

We introduce a radiance representation that is both structured and fully explicit and thus greatly facilitates 3D generative modeling. Existing radiance representations either require an implicit feature decoder, which significantly degrades the modeling power of the representation, or are spatially unstructured, making them difficult to integrate with mainstream 3D diffusion methods. We derive GaussianCube by first using a novel densification-constrained Gaussian fitting algorithm, which yields high-accuracy fitting using a fixed number of free Gaussians, and then rearranging these Gaussians into a predefined voxel grid via Optimal Transport. Since GaussianCube is a structured grid representation, it allows us to use standard 3D U-Net as our backbone in diffusion modeling without elaborate designs. More importantly, the high-accuracy fitting of the Gaussians allows us to achieve a high-quality representation with orders of magnitude fewer parameters than previous structured representations for comparable quality, ranging from one to two orders of magnitude. The compactness of GaussianCube greatly eases the difficulty of 3D generative modeling. Extensive experiments conducted on unconditional and class-conditioned object generation, digital avatar creation, and text-to-3D synthesis all show that our model achieves state-of-the-art generation results both qualitatively and quantitatively, underscoring the potential of GaussianCube as a highly accurate and versatile radiance representation for 3D generative modeling. Project page: https://gaussiancube.github.io/.