mlsanityData to 14 Oct 2026

StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation

arXiv 2305.19012 ↗First posted 30 May 2023cs.CV cs.AIcited by 25

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

The recent advancements in image-text diffusion models have stimulated research interest in large-scale 3D generative models. Nevertheless, the limited availability of diverse 3D resources presents significant challenges to learning. In this paper, we present a novel method for generating high-quality, stylized 3D avatars that utilizes pre-trained image-text diffusion models for data generation and a Generative Adversarial Network (GAN)-based 3D generation network for training. Our method leverages the comprehensive priors of appearance and geometry offered by image-text diffusion models to generate multi-view images of avatars in various styles. During data generation, we employ poses extracted from existing 3D models to guide the generation of multi-view images. To address the misalignment between poses and images in data, we investigate view-specific prompts and develop a coarse-to-fine discriminator for GAN training. We also delve into attribute-related prompts to increase the diversity of the generated avatars. Additionally, we develop a latent diffusion model within the style space of StyleGAN to enable the generation of avatars based on image inputs. Our approach demonstrates superior performance over current state-of-the-art methods in terms of visual quality and diversity of the produced avatars.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 48 of its 63 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

48 of 63 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  2. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  3. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  4. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  5. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  6. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  7. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  11. 2022Classifier-Free Diffusion Guidancecited by 5,847
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
29 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  4. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  6. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  7. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  8. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  9. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  10. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  11. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  12. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  13. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 525
  14. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  15. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  16. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  17. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  18. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 449
  19. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  20. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  21. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  22. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  23. 2018Deep Appearance Models for Face Renderingcited by 164
  24. 2021OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Associationcited by 55
  25. 2022Hierarchical Normalization for Robust Monocular Depth Estimationcited by 52
  26. 20233DAvatarGAN: Bridging Domains for Personalized Editable Avatarscited by 38
  27. 2022Diffusion Guided Domain Adaptation of Image Generatorscited by 35
  28. 2021ADNet: Leveraging Error-Bias Towards Normal Direction in Face Alignmentcited by 34
  29. 2023Learning 3D-aware Image Synthesis with Unknown Pose Distributioncited by 14

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 14; 2025: 5.

25 papers.

Sort:

By area:

  1. 2023CityDreamer: Compositional Generative Model of Unbounded 3D Citiescited by 76
  2. 2023DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatarscited by 42
  3. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  4. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  5. 2024CityCraft: A Real Crafter for 3D City Generationcited by 26
  6. 2023HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillationcited by 22
  7. 2024Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Priorcited by 20
  8. 2023DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priorscited by 19
  9. 2025AvatarArtist: Open-Domain 4D Avatarizationcited by 15
  10. 2023AI-Generated Images as Data Source: The Dawn of Synthetic Eracited by 13
  11. 2024HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformationcited by 11
  12. 2023ET3D: Efficient Text-to-3D Generation via Multi-View Distillationcited by 6
13 more
  1. 2024BodyShapeGPT: SMPL Body Shape Manipulation with LLMscited by 6
  2. 2025WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstructioncited by 6
  3. 2024LeGO: Leveraging a Surface Deformation Network for Animatable Stylized Face Generation with One Examplecited by 4
  4. 2024GCA-3D: Towards Generalized and Consistent Domain Adaptation of 3D Generatorscited by 4
  5. 2024One-shot Generative Domain Adaptation in 3D GANscited by 3
  6. 2024AniArtAvatar: Animatable 3D Art Avatar from a Single Imagecited by 2
  7. 2024Revealing Directions for Text-guided 3D Face Editingcited by 2
  8. 2024StyleDiT: A Unified Framework for Diverse Child and Partner Faces Synthesis with Style Latent Diffusion Transformercited by 2
  9. 2025Snapmoji: Instant Generation of Animatable Dual-Stylized Avatarscited by 2
  10. 2024Make VLM Recognize Visual Hallucination on Cartoon Character Image with Pose Informationcited by 1
  11. 2024SceneX: Procedural Controllable Large-scale Scene Generationcited by 1
  12. 2025StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image Translationcited by 1
  13. 2025Exploring Disentangled and Controllable Human Image Synthesis: From End-to-End to Stage-by-Stagecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Guide3D: Create 3D Avatars from Text and Image Guidance
  2. 2024Instant 3D Human Avatar Generation using Image Diffusion Models
  3. 2026Dual Diffusion Models for Multi-modal Guided 3D Avatar Generation
  4. 2024RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models
  5. 2024Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Prior
  6. 2024GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space
  7. 2024Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation
  8. 2024AvatarMMC: 3D Head Avatar Generation and Editing with Multi-Modal Conditioning