mlsanityData to 7 Oct 2026

Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions

arXiv 2211.13757 ↗First posted 24 Nov 2022cs.CVcited by 133

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Probabilistic diffusion models have achieved state-of-the-art results for image synthesis, inpainting, and text-to-image tasks. However, they are still in the early stages of generating complex 3D shapes. This work proposes Diffusion-SDF, a generative model for shape completion, single-view reconstruction, and reconstruction of real-scanned point clouds. We use neural signed distance functions (SDFs) as our 3D representation to parameterize the geometry of various signals (e.g., point clouds, 2D images) through neural networks. Neural SDFs are implicit functions and diffusing them amounts to learning the reversal of their neural network weights, which we solve using a custom modulation module. Extensive experiments show that our method is capable of both realistic unconditional generation and conditional generation from partial inputs. This work expands the domain of diffusion models from learning 2D, explicit representations, to 3D, implicit representations.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 55 of its 70 references that are in the map. First posted on arXiv 24 Nov 2022.

Builds on

55 of 70 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  2. 2022Novel View Synthesis with Diffusion Models6 Oct
  3. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  4. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
38 more
  1. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  8. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  9. 2020Neural Sparse Voxel Fieldscited by 959
  10. 2020Convolutional Occupancy Networkscited by 760
  11. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  12. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  13. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  14. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  15. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  16. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  17. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  18. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  19. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 284
  20. 2021Generating Images with Sparse Representationscited by 262
  21. 2017VAE with a VampPriorcited by 247
  22. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  23. 2022From data to functa: Your data point is a function and you can treat it like onecited by 183
  24. 2022A 3D Generative Model for Structure-Based Drug Designcited by 148
  25. 2020Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfacescited by 146
  26. 2021Modulated Periodic Activations for Generalizable Local Functional Representationscited by 140
  27. 2022Neural Wavelet-domain Diffusion for 3D Shape Generationcited by 132
  28. 2020ACRONYM: A Large-Scale Grasp Dataset Based on Simulationcited by 115
  29. 2020Cascaded Refinement Network for Point Cloud Completioncited by 112
  30. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  31. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96
  32. 2022NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fieldscited by 86
  33. 2020STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Renderingcited by 76
  34. 2020A Contrastive Learning Approach for Training Variational Autoencoder Priorscited by 65
  35. 2020Multimodal Shape Completion via Conditional Generative Adversarial Networkscited by 61
  36. 2020Adversarial Generation of Continuous Implicit Shape Representationscited by 46
  37. 2022GenSDF: Two-Stage Learning of Generalizable Signed Distance Functionscited by 30
  38. 2022Pre-train, Self-train, Distill: A simple recipe for Supersizing 3D Reconstructioncited by 26

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 24; 2024: 57; 2025: 30; 2026: 22.

133 papers.

Sort:

By area:

  1. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  2. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
  3. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  4. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  5. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  6. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  7. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  8. 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refinercited by 120
  9. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  10. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 88
  11. 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 81
  12. 2023Score-based Diffusion Models in Function Spacecited by 80
38 more
  1. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Modelscited by 79
  2. 2024Scaling Mesh Generation via Compressive Tokenizationcited by 71
  3. 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
  4. 2023SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulationcited by 66
  5. 2025OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesioncited by 60
  6. 2024ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidancecited by 59
  7. 2025HoloPart: Generative 3D Part Amodal Segmentationcited by 57
  8. 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 53
  9. 2024DiffGS: Functional Gaussian Splatting Diffusioncited by 50
  10. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
  11. 2024PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidancecited by 47
  12. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  13. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  14. 2024Binocular-Guided 3D Gaussian Splatting with View Consistency for Sparse View Synthesiscited by 41
  15. 2025SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Imagescited by 41
  16. 2024OctFusion: Octree-based Diffusion Models for 3D Shape Generationcited by 40
  17. 2024Neural Network Diffusioncited by 39
  18. 2023DiffComplete: Diffusion-based Generative 3D Shape Completioncited by 35
  19. 2024GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generationcited by 35
  20. 2024An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusioncited by 34
  21. 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistencycited by 34
  22. 2025CADCrafter: Generating Computer-Aided Design Models from Unconstrained Imagescited by 31
  23. 2023UGG: Unified Generative Graspingcited by 30
  24. 2024Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Trackingcited by 27
  25. 2025OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generationcited by 27
  26. 2024Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Planecited by 26
  27. 2024Make-A-Shape: a Ten-Million-scale 3D Shape Modelcited by 25
  28. 2024SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generationcited by 24
  29. 2024UDiFF: Generating Conditional Unsigned Distance Fields with Optimal Wavelet Diffusioncited by 23
  30. 2024ArtFormer: Controllable Generation of Diverse 3D Articulated Objectscited by 21
  31. 2024GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Detailscited by 19
  32. 2025FunDiff: Diffusion Models over Function Spaces for Physics-Informed Generative Modelingcited by 19
  33. 2024PointInfinity: Resolution-Invariant Point Diffusion Modelscited by 15
  34. 2025Recurrent Diffusion for Large-Scale Parameter Generationcited by 15
  35. 2023Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapescited by 14
  36. 2024Urban Scene Diffusion through Semantic Occupancy Mapcited by 14
  37. 2025WorldGrow: Generating Infinite 3D Worldcited by 14
  38. 2023Thin On-Sensor Nanophotonic Array Camerascited by 13

50 most cited of 133 All 133 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Mosaic-SDF for 3D Generative Models
  2. 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusion
  3. 2025Guided and Unguided Conditional Diffusion Mechanisms for Structured and Semantically-Aware 3D Point Cloud Generation
  4. 2023GECCO: Geometrically-Conditioned Point Diffusion Models
  5. 2024Diff-PCC: Diffusion-based Neural Compression for 3D Point Clouds
  6. 2025Repurposing 2D Diffusion Models for 3D Shape Completion
  7. 2025Consistency Diffusion Models for Single-Image 3D Reconstruction with Priors
  8. 2024Efficient and Scalable Point Cloud Generation with Sparse Point-Voxel Diffusion Models