mlsanityData to 7 Oct 2026

Diffusion-SDF: Text-to-Shape via Voxelized Diffusion

arXiv 2212.03293 ↗First posted 6 Dec 2022cs.CV cs.AI cs.GRcited by 128

Not on the map. Why?

Cited by 51 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

With the rising industrial attention to 3D virtual modeling technology, generating novel 3D content based on specified conditions (e.g. text) has become a hot issue. In this paper, we propose a new generative 3D modeling framework called Diffusion-SDF for the challenging task of text-to-shape synthesis. Previous approaches lack flexibility in both 3D data representation and shape generation, thereby failing to generate highly diversified 3D shapes conforming to the given text descriptions. To address this, we propose a SDF autoencoder together with the Voxelized Diffusion model to learn and generate representations for voxelized signed distance fields (SDFs) of 3D shapes. Specifically, we design a novel UinU-Net architecture that implants a local-focused inner network inside the standard U-Net architecture, which enables better reconstruction of patch-independent SDF representations. We extend our approach to further text-to-shape tasks including text-conditioned shape completion and manipulation. Experimental results show that Diffusion-SDF generates both higher quality and more diversified 3D shapes that conform well to given text descriptions when compared to previous approaches. Code is available at: https://github.com/ttlmh/Diffusion-SDF

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). Builds on 54 of its 61 references that are in the map. First posted on arXiv 6 Dec 2022.

Builds on

54 of 61 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
38 more
  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  8. 2015Variational Inference with Normalizing Flowscited by 2,161
  9. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  10. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  11. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  12. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  13. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  14. 2022Diffusion Models in Vision: A Surveycited by 1,008
  15. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  16. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  17. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  18. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  19. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  20. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  21. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  22. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  23. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  24. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  25. 2017Neural 3D Mesh Renderercited by 437
  26. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  27. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  28. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  29. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  30. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 284
  31. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  32. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  33. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  34. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  35. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  36. 2019SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimizationcited by 159
  37. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  38. 2022Towards Implicit Text-Guided 3D Shape Generationcited by 68

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 35; 2024: 56; 2025: 26; 2026: 11.

128 papers.

Sort:

By area:

  1. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  2. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  3. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  4. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  5. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  6. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  7. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  8. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  9. 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generationcited by 110
  10. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  11. 2023Text-Guided Texturing by Synchronized Multi-View Diffusioncited by 86
  12. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 78
38 more
  1. 2024Towards Realistic Scene Generation with LiDAR Diffusion Modelscited by 67
  2. 2023SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulationcited by 66
  3. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 66
  4. 2024Advances in 3D Generation: A Surveycited by 62
  5. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  6. 2024Generating CAD Code with Vision-Language Models for 3D Designscited by 61
  7. 2025OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesioncited by 60
  8. 2025HoloPart: Generative 3D Part Amodal Segmentationcited by 57
  9. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  10. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusioncited by 54
  11. 2024SemCity: Semantic Scene Generation with Triplane Diffusioncited by 53
  12. 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 53
  13. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
  14. 2023CAGE: Controllable Articulation GEnerationcited by 50
  15. 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 49
  16. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
  17. 2023Mosaic-SDF for 3D Generative Modelscited by 48
  18. 2023Posterior Distillation Samplingcited by 45
  19. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  20. 2024R3D-AD: Reconstruction via Diffusion for 3D Anomaly Detectioncited by 45
  21. 2025SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Imagescited by 41
  22. 2023Point-Cloud Completion with Pretrained Text-to-image Diffusion Modelscited by 37
  23. 2023DiffComplete: Diffusion-based Generative 3D Shape Completioncited by 35
  24. 2023GeoDream: Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generationcited by 30
  25. 2023Controllable Mind Visual Diffusion Modelcited by 29
  26. 2024Taming Data and Transformers for Audio Generationcited by 28
  27. 2025OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generationcited by 27
  28. 2024Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Planecited by 26
  29. 2024Coin3D: Controllable and Interactive 3D Assets Generation with Proxy-Guided Conditioningcited by 26
  30. 2023NAP: Neural 3D Articulation Priorcited by 25
  31. 2024Make-A-Shape: a Ten-Million-scale 3D Shape Modelcited by 25
  32. 2024SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generationcited by 24
  33. 2023DiffInDScene: Diffusion-based High-Quality 3D Indoor Scene Generationcited by 23
  34. 2023HumanLiff: Layer-wise 3D Human Generation with Diffusion Modelcited by 23
  35. 2024UDiFF: Generating Conditional Unsigned Distance Fields with Optimal Wavelet Diffusioncited by 23
  36. 2025Generating Physically Stable and Buildable Brick Structures from Textcited by 23
  37. 2024MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusioncited by 22
  38. 2024Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabricationcited by 19

50 most cited of 128 All 128 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models
  2. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models
  3. 2023Vox-E: Text-guided Voxel Editing of 3D Objects
  4. 2022Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions
  5. 2023PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion
  6. 2023DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priors
  7. 2023Generating Images with 3D Annotations Using Diffusion Models
  8. 2023Breathing New Life into 3D Assets with Generative Repainting