Part-aware Shape Generation with Latent 3D Diffusion of Neural Voxel Fields
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
This paper presents a novel latent 3D diffusion model for the generation of neural voxel fields, aiming to achieve accurate part-aware structures. Compared to existing methods, there are two key designs to ensure high-quality and accurate part-aware generation. On one hand, we introduce a latent 3D diffusion process for neural voxel fields, enabling generation at significantly higher resolutions that can accurately capture rich textural and geometric details. On the other hand, a part-aware shape decoder is introduced to integrate the part codes into the neural voxel fields, guiding the accurate part decomposition and producing high-quality rendering results. Through extensive experimentation and comparisons with state-of-the-art methods, we evaluate our approach across four different classes of data. The results demonstrate the superior generative capabilities of our proposed method in part-aware shape generation, outperforming existing state-of-the-art methods.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 57 of its 73 references that are in the map. First posted on arXiv 2 May 2024.
Builds on
57 of 73 references.
Closest ancestors · 9
Same direction, newest first
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
- 2023HoloFusion: Towards Photo-realistic 3D Generative Modeling28 Aug
- 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
- 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction13 Apr
- 2023SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulation21 Mar
- 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models26 Jan
- 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
- 2022Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions24 Nov
- 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
Other references · 48
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2023Segment Anythingcited by 8,234
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
- 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
36 more
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
- 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
- 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
- 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
- 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
- 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
- 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
- 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
- 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
- 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
- 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
- 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
- 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
- 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
- 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
- 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
- 2022VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Gridscited by 133
- 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
- 2019PQ-NET: A Generative Part Seq2Seq Network for 3D Shapescited by 119
- 2022EpiGRAF: Rethinking training of 3D GANscited by 108
- 2019Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesiscited by 104
- 2020ParSeNet: A Parametric Surface Fitting Network for 3D Point Cloudscited by 96
- 2018MeshNet: Mesh Neural Network for 3D Shape Representationcited by 95
- 2020DualSDF: Semantic Shape Manipulation using a Two-Level Representationcited by 82
- 2020PIE-NET: Parametric Inference of Point Cloud Edgescited by 67
- 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
- 2019HoloGAN: Unsupervised learning of 3D representations from natural imagescited by 59
- 2023PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervisioncited by 36
- 2023DiffFacto: Controllable Part-Based 3D Point Cloud Generation with Cross Diffusioncited by 32
- 2023Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulationcited by 27
- 2022ANISE: Assembly-based Neural Implicit Surface rEconstructioncited by 16
- 2023EXIM: A Hybrid Explicit-Implicit Representation for Text-Guided 3D Shape Generationcited by 16
- 2022SSDNeRF: Semantic Soft Decomposition of Neural Radiance Fieldscited by 9
- 2020FAME: 3D Shape Generation via Functionality-Aware Model Evolutioncited by 8
- 20203D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Face Photoscited by 7
- 2018Learning Discriminative 3D Shape Representations by View Discerning Networkscited by 5
Built on by
Citing papers per year: 2025: 4; 2026: 1.
5 papers.
- 2025MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Cloudscited by 19
- 2025Voxify3D: Pixel Art Meets Volumetric Renderingcited by 3
- 2025SuperCarver: Texture-Consistent 3D Geometry Super-Resolution for High-Fidelity Surface Detail Generationcited by 2
- 2025CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Modelcited by 2
- 2026PLSR: Progressive and Localized Super-Resolution of 3D Objects via Localized Latent Voxel Diffusioncited by 0
- 2026PLSR: Progressive and Localized Super-Resolution of 3D Objects via Localized Latent Voxel Diffusion6 Sep 2026
- 2025Voxify3D: Pixel Art Meets Volumetric Rendering8 Dec 2025
- 2025CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Model17 Nov 2025
- 2025MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds20 Aug 2025
- 2025SuperCarver: Texture-Consistent 3D Geometry Super-Resolution for High-Fidelity Surface Detail Generation12 Mar 2025
Similar, not cited
Close in wording, no citation either way
- 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models
- 2025From One to More: Contextual Part Latents for 3D Generation
- 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers
- 20213D Shape Generation and Completion through Point-Voxel Diffusion
- 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generation
- 20223D Neural Field Generation using Triplane Diffusion
- 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models
- 2025Native and Compact Structured Latents for 3D Generation