mlsanityData to 7 Oct 2026

Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape

arXiv 2305.15399 ↗First posted 24 May 2023cs.CV cs.AI cs.GRcited by 29

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Synthesizing novel 3D models that resemble the input example has long been pursued by graphics artists and machine learning researchers. In this paper, we present Sin3DM, a diffusion model that learns the internal patch distribution from a single 3D textured shape and generates high-quality variations with fine geometry and texture details. Training a diffusion model directly in 3D would induce large memory and computational cost. Therefore, we first compress the input into a lower-dimensional latent space and then train a diffusion model on it. Specifically, we encode the input 3D textured shape into triplane feature maps that represent the signed distance and texture fields of the input. The denoising network of our diffusion model has a limited receptive field to avoid overfitting, and uses triplane-aware 2D convolution blocks to improve the result quality. Aside from randomly generating new samples, our model also facilitates applications such as retargeting, outpainting and local editing. Through extensive qualitative and quantitative evaluation, we show that our method outperforms prior methods in generation quality of 3D shapes.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 63 of its 118 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

63 of 118 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Images29 Mar
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  3. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generation9 Mar
  4. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  5. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  6. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  7. 20223D Neural Field Generation using Triplane Diffusion30 Nov
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  9. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation17 Nov
  10. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  11. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 52

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  12. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
38 more
  1. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  2. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  3. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  4. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  5. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  6. 2020Convolutional Occupancy Networkscited by 760
  7. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  8. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  9. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  10. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  11. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  12. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  13. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  14. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  15. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  16. 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 266
  17. 2020Learning Gradient Fields for Shape Generationcited by 245
  18. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 205
  19. 2022SINE: SINgle Image Editing with Text-to-Image Diffusion Modelscited by 159
  20. 2022SolidGen: An Autoregressive Model for Direct B-rep Synthesiscited by 78
  21. 2020Improved Techniques for Training Single-Image GANscited by 77
  22. 2018Non-Stationary Texture Synthesis by Adversarial Expansioncited by 66
  23. 2023Single Motion Diffusioncited by 65
  24. 2022SinDDM: A Single Image Denoising Diffusion Modelcited by 60
  25. 2022SinFusion: Training Diffusion Models on a Single Image or Videocited by 58
  26. 2020Deep Geometric Texture Synthesiscited by 51
  27. 2021Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Modelscited by 51
  28. 2020DECOR-GAN: 3D Shape Detailization by Conditional Refinementcited by 45
  29. 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
  30. 2022SinDiffusion: Learning a Diffusion Model from a Single Natural Imagecited by 42
  31. 2022Learning to Generate 3D Shapes from a Single Examplecited by 31
  32. 2022SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Mapscited by 19
  33. 2021Catch-A-Waveform: Learning to Generate Audio from a Single Short Examplecited by 15
  34. 2023Patch-based 3D Natural Scene Generation from a Single Examplecited by 15
  35. 2021Diverse Generation from a Single Video Made Possiblecited by 13
  36. 2021ExSinGAN: Learning an Explainable Generative Model from a Single Imagecited by 12
  37. 2022SinGRAF: Learning a 3D Generative Radiance Field for a Single Scenecited by 11
  38. 2023Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflectioncited by 10

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 12; 2025: 7; 2026: 5.

29 papers.

Sort:

By area:

  1. 2023Improving Tuning-Free Real Image Editing with Proximal Guidancecited by 87
  2. 2024pix2gestalt: Amodal Segmentation by Synthesizing Wholescited by 70
  3. 2024SemCity: Semantic Scene Generation with Triplane Diffusioncited by 53
  4. 2024Controlling the World by Sleight of Handcited by 19
  5. 2024ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplarscited by 14
  6. 2023Single Mesh Diffusion Models with Field Latents for Texture Generationcited by 11
  7. 2024TexRO: Generating Delicate Textures of 3D Models by Recursive Optimizationcited by 9
  8. 2023Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Modelscited by 8
  9. 2024CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesiscited by 6
  10. 2026PEGAsus: 3D Personalization of Geometry and Appearancecited by 5
  11. 2024Blaze3DM: Marry Triplane Representation with Diffusion for 3D Medical Inverse Problem Solvingcited by 4
  12. 2024EraseDraw: Learning to Draw Step-by-Step via Erasing Objects from Imagescited by 4
17 more
  1. 2024UV-free Texture Generation with Denoising and Geodesic Heat Diffusionscited by 3
  2. 2025ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusioncited by 3
  3. 2023Retargeting Visual Data with Deformation Fieldscited by 2
  4. 2025TriTex: Learning Texture from a Single Mesh via Triplane Semantic Featurescited by 2
  5. 2025Category-Aware 3D Object Composition with Disentangled Texture and Shape Multi-view Diffusioncited by 2
  6. 2026RefAny3D: 3D Asset-Referenced Diffusion Models for Image Generationcited by 2
  7. 2023GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learningcited by 1
  8. 2024Towards Practical Single-shot Motion Synthesiscited by 1
  9. 2025DreamTexture: Shape from Virtual Texture with Analysis by Augmentationcited by 1
  10. 2025Controllable Single-shot Animation Blending with Temporal Conditioningcited by 1
  11. 2026StructDiff: A Structure-Preserving and Spatially Controllable Diffusion Model for Single-Image Generationcited by 1
  12. 2024Factorized Video Autoencoders for Efficient Generative Modellingcited by 0
  13. 2024ExCellGen: Fast, Controllable, Photorealistic 3D Scene Generation from a Single Real-World Exemplarcited by 0
  14. 20253D Shape Generation: A Surveycited by 0
  15. 2025ReSWD: ReSTIR'd, not shaken. Combining Reservoir Sampling and Sliced Wasserstein Distance for Variance Reductioncited by 0
  16. 2026FurnSet: Exploiting Repeats for 3D Scene Reconstructioncited by 0
  17. 2026Rethinking 3D Shape Generation: Diffusion over Superquadricscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior
  2. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models
  3. 2025Textured 3D Regenerative Morphing with 3D Diffusion Prior
  4. 2023Breathing New Life into 3D Assets with Generative Repainting
  5. 2025DoubleDiffusion: Combining Heat Diffusion with Denoising Diffusion for Texture Generation on 3D Meshes
  6. 20213D Shape Generation and Completion through Point-Voxel Diffusion
  7. 2023PolyDiff: Generating 3D Polygonal Meshes with Diffusion Models
  8. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion