mlsanityData to 7 Oct 2026

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion

arXiv 2212.06135 ↗First posted 12 Dec 2022cs.CVcited by 309

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

This paper presents a 3D generative model that uses diffusion models to automatically generate 3D digital avatars represented as neural radiance fields. A significant challenge in generating such avatars is that the memory and processing costs in 3D are prohibitive for producing the rich details required for high-quality avatars. To tackle this problem we propose the roll-out diffusion network (Rodin), which represents a neural radiance field as multiple 2D feature maps and rolls out these maps into a single 2D feature plane within which we perform 3D-aware diffusion. The Rodin model brings the much-needed computational efficiency while preserving the integrity of diffusion in 3D by using 3D-aware convolution that attends to projected features in the 2D feature plane according to their original relationship in 3D. We also use latent conditioning to orchestrate the feature generation for global coherence, leading to high-fidelity avatars and enabling their semantic editing based on text prompts. Finally, we use hierarchical synthesis to further enhance details. The 3D avatars generated by our model compare favorably with those produced by existing generative techniques. We can generate highly detailed avatars with realistic hairstyles and facial hair like beards. We also demonstrate 3D avatar generation from image or text as well as text-guided editability.

Key facts. Cited by 309 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 69 of its 83 references that are in the map. First posted on arXiv 12 Dec 2022.

Builds on

69 of 83 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  2. 2022Novel View Synthesis with Diffusion Models6 Oct
  3. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  4. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 65

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2022Generative Adversarial Networkscited by 5,603
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
38 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  9. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  13. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  14. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  15. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  16. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  17. 2019An Introduction to Variational Autoencoderscited by 997
  18. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  19. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  20. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  21. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  22. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  23. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  24. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  25. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  26. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  27. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  28. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  29. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  30. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  31. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 347
  32. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  33. 2020Learning Gradient Fields for Shape Generationcited by 245
  34. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  35. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  36. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  37. 2018Deep Appearance Models for Face Renderingcited by 164
  38. 2022Pretraining is All You Need for Image-to-Image Translationcited by 160

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 92; 2024: 121; 2025: 65; 2026: 31.

309 papers.

Sort:

By area:

  1. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  2. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  3. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  4. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  5. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  6. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  7. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  8. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  9. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  10. 2023Text-to-3D using Gaussian Splattingcited by 299
  11. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  12. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
38 more
  1. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  2. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  3. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  4. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  5. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
  6. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  7. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  8. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  9. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145
  10. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  11. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  12. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  13. 2023TADA! Text to Animatable Digital Avatarscited by 109
  14. 2023DemoFusion: Democratising High-Resolution Image Generation With No cited by 105
  15. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humanscited by 104
  16. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  17. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 100
  18. 2024Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoderscited by 93
  19. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  20. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  21. 2023HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generationcited by 88
  22. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
  23. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 86
  24. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 84
  25. 2025Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modelingcited by 84
  26. 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 81
  27. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusioncited by 72
  28. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
  29. 2024Scaling Mesh Generation via Compressive Tokenizationcited by 71
  30. 2023Structure-Aware Sparse-View X-ray 3D Reconstructioncited by 70
  31. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  32. 2023Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priorscited by 64
  33. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Modelscited by 64
  34. 2024Advances in 3D Generation: A Surveycited by 62
  35. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  36. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  37. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  38. 2024CG-SLAM: Efficient Dense RGB-D SLAM in a Consistent Uncertainty-aware 3D Gaussian Fieldcited by 61

50 most cited of 309 All 309 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Instant 3D Human Avatar Generation using Image Diffusion Models
  2. 2023DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priors
  3. 2024Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models
  4. 2024Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy
  5. 2023Breathing New Life into 3D Assets with Generative Repainting
  6. 2024AvatarMMC: 3D Head Avatar Generation and Editing with Multi-Modal Conditioning
  7. 2026Dual Diffusion Models for Multi-modal Guided 3D Avatar Generation
  8. 2025Hyper Diffusion Avatars: Dynamic Human Avatar Generation using Network Weight Space Diffusion