mlsanityData to 7 Oct 2026

Single-Image 3D Human Digitization with Shape-Guided Diffusion

arXiv 2311.09221 ↗First posted 15 Nov 2023cs.CVcited by 42

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present an approach to generate a 360-degree view of a person with a consistent, high-resolution appearance from a single input image. NeRF and its variants typically require videos or images from different viewpoints. Most existing approaches taking monocular input either rely on ground-truth 3D scans for supervision or lack 3D consistency. While recent 3D generative models show promise of 3D consistent human digitization, these approaches do not generalize well to diverse clothing appearances, and the results lack photorealism. Unlike existing work, we utilize high-capacity 2D diffusion models pretrained for general image synthesis tasks as an appearance prior of clothed humans. To achieve better 3D consistency while retaining the input identity, we progressively synthesize multiple views of the human in the input image by inpainting missing regions with shape-guided diffusion conditioned on silhouette and surface normal. We then fuse these synthesized multi-view images via inverse rendering to obtain a fully textured high-resolution 3D mesh of the given person. Experiments show that our approach outperforms prior methods and achieves photorealistic 360-degree synthesis of a wide range of clothed humans with complex textures from a single image.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 72 of its 86 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

72 of 86 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  3. 20233D-aware Image Generation using 2D Diffusion Models31 Mar
  4. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  5. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  6. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  7. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  8. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  10. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 62

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  6. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  7. 2017End-to-end Recovery of Human Shape and Posecited by 883
  8. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  9. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  10. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  11. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  12. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
38 more
  1. 2021Neural Fields in Visual Computing and Beyondcited by 539
  2. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  3. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  4. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  5. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  6. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  7. 2018Everybody Dance Nowcited by 313
  8. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  9. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  10. 2017Pose Guided Person Image Generationcited by 251
  11. 2021Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensorscited by 246
  12. 2019Multi-Garment Net: Learning to Dress 3D People from Imagescited by 244
  13. 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
  14. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  15. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  16. 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
  17. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  18. 2022ECON: Explicit Clothed humans Optimized via Normal integrationcited by 199
  19. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  20. 2021ARCH++: Animation-Ready Clothed Human Reconstruction Revisitedcited by 174
  21. 2019Learning to Reconstruct People in Clothing from a Single RGB Cameracited by 168
  22. 2017Deformable GANs for Pose-based Human Image Generationcited by 166
  23. 2019Tex2Shape: Detailed Full Human Body Geometry From a Single Imagecited by 166
  24. 2021Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Renderingcited by 158
  25. 2021SMPLicit: Topology-aware Generative Model for Clothed Peoplecited by 154
  26. 2020Controllable Person Image Synthesis with Attribute-Decomposed GANcited by 150
  27. 2018BodyNet: Volumetric Inference of 3D Human Body Shapescited by 141
  28. 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
  29. 2022Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothingcited by 126
  30. 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
  31. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  32. 2020Deep Image Spatial Transformation for Person Image Generationcited by 100
  33. 2017Disentangled Person Image Generationcited by 98
  34. 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
  35. 2021Driving-Signal Aware Full-Body Avatarscited by 81
  36. 2022KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypointscited by 77
  37. 2018SiCloPe: Silhouette-Based Clothed Peoplecited by 68
  38. 2019Moulding Humans: Non-parametric 3D Human Shape Estimation from Single Imagescited by 67

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 16; 2025: 18; 2026: 6.

42 papers.

Sort:

By area:

  1. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 271
  2. 2023SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstructioncited by 77
  3. 2023SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusioncited by 72
  4. 2024Advances in 3D Generation: A Surveycited by 62
  5. 2024IDOL: Instant Photorealistic 3D Human Creation from a Single Imagecited by 55
  6. 2024HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priorscited by 54
  7. 2024TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusioncited by 51
  8. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
  9. 2024Generalizable Human Gaussians from Single-View Imagecited by 29
  10. 2024Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRMcited by 23
  11. 2024GALA: Generating Animatable Layered Assets from a Single Scancited by 21
  12. 2024DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Posescited by 16
30 more
  1. 2024Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Priorcited by 15
  2. 2024Instant 3D Human Avatar Generation using Image Diffusion Modelscited by 14
  3. 2025UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collectionscited by 10
  4. 2025HumanRAM: Feed-forward Human Reconstruction and Animation Model using Transformerscited by 9
  5. 2025MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attentioncited by 8
  6. 2025AdaHuman: Animatable Detailed 3D Human Generation with Compositional Multiview Diffusioncited by 8
  7. 2025HumanGif: Single-View Human Diffusion with Generative Priorcited by 7
  8. 2025Target-Aware Video Diffusion Modelscited by 7
  9. 2024GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Datacited by 6
  10. 2025WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstructioncited by 6
  11. 2025CHROME: Clothed Human Reconstruction with Occlusion-Resilience and Multiview-Consistency from a Single Imagecited by 6
  12. 2024Single Image, Any Face: Generalisable 3D Face Generationcited by 4
  13. 2025DeClotH: Decomposable 3D Cloth and Human Body Reconstruction from a Single Imagecited by 4
  14. 2025Dream, Lift, Animate: From Single Images to Animatable Gaussian Avatarscited by 4
  15. 2024Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatarscited by 3
  16. 2025R3-Avatar: Record and Retrieve Temporal Codebook for Reconstructing Photorealistic Human Avatarscited by 3
  17. 2025EverybodyDance: Bipartite Graph-Based Identity Correspondence for Multi-Character Animationcited by 3
  18. 2025SVAD: From Single Image to 3D Avatar via Synthetic Data Generation with Video Diffusion and Data Augmentationcited by 2
  19. 2025PARTE: Part-Guided Texturing for 3D Human Reconstruction from a Single Imagecited by 2
  20. 2024SOAR: Self-Occluded Avatar Recovery from a Single Video In the Wildcited by 1
  21. 2025MVD-HuGaS: Human Gaussians from a Single Image via 3D Human Multi-view Diffusion Priorcited by 1
  22. 2025Learning Efficient and Generalizable Human Representation with Human Gaussian Modelcited by 1
  23. 2025DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspectivecited by 0
  24. 2025Dream3DAvatar: Text-Controlled 3D Avatar Reconstruction from a Single Imagecited by 0
  25. 2026Joint Geometry-Appearance Human Reconstruction in a Unified Latent Space via Bridge Diffusioncited by 0
  26. 2026PoseGaussian: Pose-Driven Novel View Synthesis for Robust 3D Human Reconstructioncited by 0
  27. 2026Direct Reward Fine-Tuning on Poses for Single Image to 3D Human in the Wildcited by 0
  28. 2026DogWeave: High-Fidelity 3D Canine Reconstruction from a Single Image via Normal Fusion and Conditional Inpaintingcited by 0
  29. 2026HumanNOVA: Photorealistic, Universal and Rapid 3D Human Avatar Modeling from a Single Imagecited by 0
  30. 2026DiGS-Avatar: Single-Image Animatable 3D Human Reconstruction via UV-Space Diffusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Synthesizing Moving People with 3D Control
  2. 2023Chupa: Carving 3D Clothed Humans from Skinned Shape Priors using 2D Diffusion Probabilistic Models
  3. 2023HumanLiff: Layer-wise 3D Human Generation with Diffusion Model
  4. 2023HumanRef: Single Image to 3D Human Generation via Reference-Guided Diffusion
  5. 2024PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing
  6. 2025Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinates
  7. 2024Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models
  8. 2026Spatio-Temporal Garment Reconstruction Using Diffusion Mapping via Pattern Coordinates