mlsanityData to 14 Oct 2026

DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models

arXiv 2304.00916 ↗First posted 3 Apr 2023cs.CVcited by 154

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present DreamAvatar, a text-and-shape guided framework for generating high-quality 3D human avatars with controllable poses. While encouraging results have been reported by recent methods on text-guided 3D common object generation, generating high-quality human avatars remains an open challenge due to the complexity of the human body's shape, pose, and appearance. We propose DreamAvatar to tackle this challenge, which utilizes a trainable NeRF for predicting density and color for 3D points and pretrained text-to-image diffusion models for providing 2D self-supervision. Specifically, we leverage the SMPL model to provide shape and pose guidance for the generation. We introduce a dual-observation-space design that involves the joint optimization of a canonical space and a posed space that are related by a learnable deformation field. This facilitates the generation of more complete textures and geometry faithful to the target pose. We also jointly optimize the losses computed from the full body and from the zoomed-in 3D head to alleviate the common multi-face ''Janus'' problem and improve facial details in the generated avatars. Extensive evaluations demonstrate that DreamAvatar significantly outperforms existing methods, establishing a new state-of-the-art for text-and-shape guided 3D human avatar generation.

Key facts. Cited by 154 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 51 of its 72 references that are in the map. First posted on arXiv 3 Apr 2023.

Builds on

51 of 72 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  3. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  4. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec
  5. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  7. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  8. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  9. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  8. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  9. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  10. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  11. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  12. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
30 more
  1. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  2. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  3. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  4. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  5. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  6. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 575
  7. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  8. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  9. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  10. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  11. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  12. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  13. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
  14. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
  15. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 334
  16. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  17. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  18. 2021Neural Articulated Radiance Fieldcited by 196
  19. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  20. 2022StyleGAN-Human: A Data-Centric Odyssey of Human Generationcited by 144
  21. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  22. 2013Stabilitycited by 120
  23. 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
  24. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  25. 2022Generative Neural Articulated Radiance Fieldscited by 101
  26. 2022Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representationscited by 65
  27. 2022MPS-NeRF: Generalizable 3D Human Rendering from Multiview Imagescited by 53
  28. 2022HumanGen: Generating Human Radiance Fields with Explicit Priorscited by 33
  29. 2020Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Datacited by 30
  30. 2022JIFF: Jointly-aligned Implicit Face Function for High Quality Single View Clothed Human Reconstructioncited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 52; 2024: 59; 2025: 30; 2026: 13.

154 papers.

Sort:

By area:

  1. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  2. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 279
  3. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
  4. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 272
  5. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  6. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  7. 2023TADA! Text to Animatable Digital Avatarscited by 110
  8. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humanscited by 104
  9. 2023DreamHuman: Animatable 3D Avatars from Textcited by 102
  10. 2024Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoderscited by 98
  11. 2023HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splattingcited by 89
  12. 2023HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generationcited by 88
38 more
  1. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  2. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  3. 2023DreamWaltz: Make a Scene with Complex 3D Animatable Avatarscited by 82
  4. 2023SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusioncited by 72
  5. 2023ChatPose: Chatting about 3D Human Posecited by 72
  6. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusioncited by 72
  7. 2024View-Consistent 3D Editing with Gaussian Splattingcited by 67
  8. 2023Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priorscited by 64
  9. 2023GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learningcited by 63
  10. 2024Advances in 3D Generation: A Surveycited by 63
  11. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  12. 2023Gaussian Shell Maps for Efficient 3D Human Generationcited by 61
  13. 2025LHM: Large Animatable Human Reconstruction Model from a Single Image in Secondscited by 61
  14. 2023AvatarVerse: High-quality & Stable 3D Avatar Creation from Text and Posecited by 60
  15. 2023HeadSculpt: Crafting 3D Head Avatars with Textcited by 49
  16. 2023DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatarscited by 42
  17. 2023Text-Guided Generation and Editing of Compositional 3D Avatarscited by 39
  18. 2023AvatarBooth: High-Quality and Customizable 3D Human Avatar Generationcited by 37
  19. 2024AniGS: Animatable Gaussian Avatar from a Single Image with Inconsistent Gaussian Reconstructioncited by 36
  20. 2024GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusioncited by 36
  21. 2024HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splattingcited by 35
  22. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  23. 2023Disentangled Clothed Avatar Generation from Text Descriptionscited by 33
  24. 2023Control4D: Efficient 4D Portrait Editing with Textcited by 32
  25. 2023HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstylescited by 30
  26. 2024Generalizable Human Gaussians from Single-View Imagecited by 29
  27. 2024CharacterGen: Efficient 3D Character Generation from Single Images with Multi-View Pose Canonicalizationcited by 28
  28. 2023Stable Score Distillation for High-Quality 3D Generationcited by 27
  29. 2025DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimizationcited by 27
  30. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animationcited by 25
  31. 2023SEEAvatar: Photorealistic Text-to-3D Avatar Generation with Constrained Geometry and Appearancecited by 24
  32. 2024DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characterscited by 24
  33. 2024Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creationcited by 23
  34. 2023HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillationcited by 22
  35. 2024GALA: Generating Animatable Layered Assets from a Single Scancited by 21
  36. 2024TELA: Text to Layer-wise 3D Clothed Human Generationcited by 20
  37. 2023DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priorscited by 19
  38. 2024Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motioncited by 19

50 most cited of 154 All 154 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Instant 3D Human Avatar Generation using Image Diffusion Models
  2. 2022AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars
  3. 2026Dual Diffusion Models for Multi-modal Guided 3D Avatar Generation
  4. 2023AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control
  5. 2025Multimodal Generation of Animatable 3D Human Models with AvatarForge
  6. 2025GAS: Generative Avatar Synthesis from a Single Image
  7. 2023Reality's Canvas, Language's Brush: Crafting 3D Avatars from Monocular Video
  8. 2023Text-Conditional Contextualized Avatars For Zero-Shot Personalization