mlsanityData to 14 Oct 2026

Chupa: Carving 3D Clothed Humans from Skinned Shape Priors using 2D Diffusion Probabilistic Models

arXiv 2305.11870 ↗First posted 19 May 2023cs.CVcited by 20

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We propose a 3D generation pipeline that uses diffusion models to generate realistic human digital avatars. Due to the wide variety of human identities, poses, and stochastic details, the generation of 3D human meshes has been a challenging problem. To address this, we decompose the problem into 2D normal map generation and normal map-based 3D reconstruction. Specifically, we first simultaneously generate realistic normal maps for the front and backside of a clothed human, dubbed dual normal maps, using a pose-conditional diffusion model. For 3D reconstruction, we "carve" the prior SMPL-X mesh to a detailed 3D mesh according to the normal maps through mesh optimization. To further enhance the high-frequency details, we present a diffusion resampling scheme on both body and facial regions, thus encouraging the generation of realistic digital avatars. We also seamlessly incorporate a recent text-to-image diffusion model to support text-based human identity control. Our method, namely, Chupa, is capable of generating realistic 3D clothed humans with better perceptual quality and identity variety.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 78 of its 104 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

78 of 104 references.

Closest ancestors · 8

Same direction, newest first

  1. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec
  2. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  3. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  4. 20223D Neural Field Generation using Triplane Diffusion30 Nov
  5. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  6. 2022Novel View Synthesis with Diffusion Models6 Oct
  7. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  8. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan

Other references · 70

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  12. 2022Classifier-Free Diffusion Guidancecited by 5,847
38 more
  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  9. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  10. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  11. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  12. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  13. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  14. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  15. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  16. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  17. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  18. 2017End-to-end Recovery of Human Shape and Posecited by 885
  19. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  20. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  21. 2020Convolutional Occupancy Networkscited by 761
  22. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  23. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  24. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  25. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  26. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  27. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
  28. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  29. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  30. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  31. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  32. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  33. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  34. 2021Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensorscited by 246
  35. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  36. 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
  37. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  38. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 205

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 10; 2025: 4; 2026: 1.

20 papers.

Sort:

By area:

  1. 2023TADA! Text to Animatable Digital Avatarscited by 110
  2. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humanscited by 104
  3. 2024Advances in 3D Generation: A Surveycited by 63
  4. 2023Disentangled Clothed Avatar Generation from Text Descriptionscited by 33
  5. 2024PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshingcited by 23
  6. 2024GALA: Generating Animatable Layered Assets from a Single Scancited by 21
  7. 2024GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Detailscited by 19
  8. 2024Instant 3D Human Avatar Generation using Image Diffusion Modelscited by 14
  9. 2024CanonicalFusion: Generating Drivable 3D Human Avatars from Multiple Imagescited by 12
  10. 2024Diffusion Models in 3D Vision: A Surveycited by 11
  11. 2025UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collectionscited by 10
  12. 2025Disentangled Clothed Avatar Generation with Layered Representationcited by 7
8 more
  1. 2023Joint2Human: High-quality 3D Human Generation via Compact Spherical Embedding of 3D Jointscited by 6
  2. 2023MVHuman: Tailoring 2D Diffusion with Multi-view Sampling For Realistic 3D Human Generationcited by 6
  3. 2025WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstructioncited by 6
  4. 2024InstructHumans: Editing Animated 3D Human Textures with Instructionscited by 4
  5. 2024Progress and Prospects in 3D Generative AI: A Technical Overview including 3D humancited by 2
  6. 2024FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modelingcited by 2
  7. 2025PARTE: Part-Guided Texturing for 3D Human Reconstruction from a Single Imagecited by 2
  8. 2026TGRHuman: Text-Guided Realistic 3D Human Generation via Diffusion Renderercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models
  2. 2023Single-Image 3D Human Digitization with Shape-Guided Diffusion
  3. 2024DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans
  4. 2024Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy
  5. 2023AvatarFusion: Zero-shot Generation of Clothing-Decoupled 3D Avatars Using 2D Diffusion
  6. 2024Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation
  7. 2023HumanLiff: Layer-wise 3D Human Generation with Diffusion Model
  8. 2026Spatio-Temporal Garment Reconstruction Using Diffusion Mapping via Pattern Coordinates