mlsanityData to 7 Oct 2026

3D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models

arXiv 2212.00842 ↗First posted 1 Dec 2022cs.CVcited by 67

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Diffusion models have shown great promise for image generation, beating GANs in terms of generation diversity, with comparable image quality. However, their application to 3D shapes has been limited to point or voxel representations that can in practice not accurately represent a 3D surface. We propose a diffusion model for neural implicit representations of 3D shapes that operates in the latent space of an auto-decoder. This allows us to generate diverse and high quality 3D surfaces. We additionally show that we can condition our model on images or text to enable image-to-3D generation and text-to-3D generation using CLIP embeddings. Furthermore, adding noise to the latent codes of existing shapes allows us to explore shape variations.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 59 of its 72 references that are in the map. First posted on arXiv 1 Dec 2022.

Builds on

59 of 72 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  2. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  3. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
38 more
  1. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  4. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  6. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  7. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  8. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  9. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  10. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  11. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  12. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  13. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  14. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  15. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  16. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  17. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  18. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  19. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  20. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  21. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  22. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  23. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  24. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 275
  25. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  26. 2020Learning Gradient Fields for Shape Generationcited by 245
  27. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  28. 2021Shape As Points: A Differentiable Poisson Solvercited by 186
  29. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  30. 2020DiffusionNet: Discretization Agnostic Learning on Surfacescited by 163
  31. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  32. 2022Neural Wavelet-domain Diffusion for 3D Shape Generationcited by 132
  33. 2016Generative and Discriminative Voxel Modeling with Convolutional Neural Networkscited by 128
  34. 2020MeshSDF: Differentiable Iso-Surface Extractioncited by 112
  35. 2017MarrNet: 3D Shape Reconstruction via 2.5D Sketchescited by 108
  36. 2018PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attentioncited by 105
  37. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 97
  38. 2020Discrete Point Flow Networks for Efficient Point Cloud Generationcited by 87

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 28; 2024: 26; 2025: 9; 2026: 4.

67 papers.

Sort:

By area:

  1. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  2. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  3. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  4. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  5. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  6. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  7. 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generationcited by 110
  8. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 100
  9. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 86
  10. 2023Text-Guided Texturing by Synchronized Multi-View Diffusioncited by 86
  11. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
  12. 2023SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulationcited by 66
38 more
  1. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  2. 2024Advances in 3D Generation: A Surveycited by 62
  3. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  4. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  5. 2024Part123: Part-aware 3D Reconstruction from a Single-view Imagecited by 54
  6. 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 53
  7. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
  8. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  9. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  10. 2023VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encodercited by 39
  11. 2023DiffComplete: Diffusion-based Generative 3D Shape Completioncited by 35
  12. 2023DreamSparse: Escaping from Plato's Cave with 2D Frozen Diffusion Model Given Sparse Viewscited by 31
  13. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  14. 2023NAP: Neural 3D Articulation Priorcited by 25
  15. 2024TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and Resamplingcited by 24
  16. 2023HumanLiff: Layer-wise 3D Human Generation with Diffusion Modelcited by 23
  17. 2024UDiFF: Generating Conditional Unsigned Distance Fields with Optimal Wavelet Diffusioncited by 23
  18. 2025Generating Physically Stable and Buildable Brick Structures from Textcited by 23
  19. 2025LL3M: Large Language 3D Modelerscited by 18
  20. 2024DDMI: Domain-Agnostic Latent Diffusion Models for Synthesizing High-Quality Implicit Neural Representationscited by 16
  21. 2024PointInfinity: Resolution-Invariant Point Diffusion Modelscited by 15
  22. 2023Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapescited by 14
  23. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
  24. 2024Topology-Aware Latent Diffusion for 3D Shape Generationcited by 13
  25. 20233D VR Sketch Guided 3D Shape Prototyping and Explorationcited by 12
  26. 2023NVS-Adapter: Plug-and-Play Novel View Synthesis from a Single Imagecited by 11
  27. 2024Diffusion Models in 3D Vision: A Surveycited by 11
  28. 2023MVDD: Multi-View Depth Diffusion Modelscited by 10
  29. 2023WildFusion: Learning 3D-Aware Latent Diffusion Models in View Spacecited by 8
  30. 2024GetMesh: A Controllable Model for High-quality Mesh Generation and Manipulationcited by 8
  31. 2024DreamCraft3D++: Efficient Hierarchical 3D Generation with Multi-Plane Reconstruction Modelcited by 7
  32. 2025Towards Learning to Complete Anything in Lidarcited by 7
  33. 2023Hybrid Neural Diffeomorphic Flow for Shape Representation and Generation via Triplanecited by 6
  34. 2025Representing 3D Shapes With 64 Latent Vectors for 3D Diffusion Modelscited by 6
  35. 2024Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generationcited by 5
  36. 2024Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Imagescited by 5
  37. 2026Learn2Fold: Structured Origami Generation with World Model Planningcited by 5
  38. 2023Text-guided 3D Human Generation from 2D Collectionscited by 4

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Part-aware Shape Generation with Latent 3D Diffusion of Neural Voxel Fields
  2. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models
  3. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models
  4. 20223D Neural Field Generation using Triplane Diffusion
  5. 2022Deep Generative Models on 3D Representations: A Survey
  6. 2022IC3D: Image-Conditioned 3D Diffusion for Shape Generation
  7. 2023Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models
  8. 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusion