mlsanityData to 7 Oct 2026

High-Fidelity 3D Face Generation from Natural Language Descriptions

arXiv 2305.03302 ↗First posted 5 May 2023cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

Synthesizing high-quality 3D face models from natural language descriptions is very valuable for many applications, including avatar creation, virtual reality, and telepresence. However, little research ever tapped into this task. We argue the major obstacle lies in 1) the lack of high-quality 3D face data with descriptive text annotation, and 2) the complex mapping relationship between descriptive language space and shape/appearance space. To solve these problems, we build Describe3D dataset, the first large-scale dataset with fine-grained text descriptions for text-to-3D face generation task. Then we propose a two-stage framework to first generate a 3D face that matches the concrete descriptions, then optimize the parameters in the 3D shape and texture space with abstract description to refine the 3D face model. Extensive experimental results show that our method can produce a faithful 3D face that conforms to the input descriptions with higher accuracy and quality than previous methods. The code and Describe3D dataset are released at https://github.com/zhuhao-nju/describe3d .

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 45 of its 60 references that are in the map. First posted on arXiv 5 May 2023.

Builds on

45 of 60 references.

References · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2022Generative Adversarial Networkscited by 5,603
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  10. 2016Generative Adversarial Text to Image Synthesiscited by 735
  11. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  12. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
33 more
  1. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  2. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  3. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  4. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  5. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  6. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  7. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  8. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  9. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  10. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  11. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  12. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  13. 2018Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Networkcited by 195
  14. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  15. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  16. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  17. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  18. 2016Learning What and Where to Drawcited by 105
  19. 2018Inferring Semantic Layout for Hierarchical Text-to-Image Synthesiscited by 101
  20. 2018MeshNet: Mesh Neural Network for 3D Shape Representationcited by 95
  21. 2021MoFaNeRF: Morphable Facial Neural Radiance Fieldcited by 94
  22. 2017Semantic Image Synthesis via Adversarial Learningcited by 87
  23. 2022Towards Implicit Text-Guided 3D Shape Generationcited by 68
  24. 2022ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representationscited by 64
  25. 2019Semantics Disentangling for Text-to-Image Generationcited by 61
  26. 2022AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatarscited by 55
  27. 2022Text and Image Guided 3D Avatar Generation and Manipulationcited by 35
  28. 2019Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networkscited by 34
  29. 2021FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstructioncited by 34
  30. 2022AnyFace: Free-style Text-to-Face Synthesis and Manipulationcited by 33
  31. 2022Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Functioncited by 14
  32. 2020Age-Oriented Face Synthesis with Conditional Discriminator Pool and Adversarial Triplet Losscited by 6
  33. 2023RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 15; 2025: 4; 2026: 5.

32 papers.

Sort:

By area:

  1. 2024IDOL: Instant Photorealistic 3D Human Creation from a Single Imagecited by 55
  2. 2023Text-Guided Generation and Editing of Compositional 3D Avatarscited by 39
  3. 2023AvatarBooth: High-Quality and Customizable 3D Human Avatar Generationcited by 37
  4. 2024FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Videocited by 31
  5. 2023Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detailcited by 23
  6. 2024Mani-GS: Gaussian Splatting Manipulation with Triangular Meshcited by 22
  7. 2023HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillationcited by 20
  8. 2024MeshUp: Multi-Target Mesh Deformation via Blended Score Distillationcited by 17
  9. 2024A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generationcited by 13
  10. 2024AWOL: Analysis WithOut synthesis using Languagecited by 12
  11. 2023Controllable 3D Face Generation with Conditional Style Code Diffusioncited by 9
  12. 2024Words in Motion: Extracting Interpretable Control Vectors for Motion Transformerscited by 9
20 more
  1. 2023Text-Guided 3D Face Synthesis -- From Generation to Editingcited by 8
  2. 2024Fast Text-to-3D-Aware Face Generation and Manipulation via Direct Cross-modal Mapping and Geometric Regularizationcited by 8
  3. 2024Head360: Learning a Parametric 3D Full-Head for Free-View Synthesis in 360degcited by 8
  4. 2024MMoFusion: Multi-modal Co-Speech Motion Generation with Diffusion Modelcited by 5
  5. 2024Towards Native Generative Model for 3D Head Avatarcited by 5
  6. 2023Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distributioncited by 4
  7. 2024FaceGPT: Self-supervised Learning to Chat about 3D Human Facescited by 4
  8. 2024GenCA: A Text-conditioned Generative Model for Realistic and Drivable Codec Avatarscited by 3
  9. 2024StrandHead: Text to Hair-Disentangled 3D Head Avatars Using Human-Centric Priorscited by 3
  10. 2025Evaluating Deepfake Detectors in the Wildcited by 3
  11. 2026Spherical Geometry Diffusion: Generating High-quality 3D Face Geometry via Sphere-anchored Representationscited by 1
  12. 2023Reality's Canvas, Language's Brush: Crafting 3D Avatars from Monocular Videocited by 0
  13. 2024Multimodal Conditional 3D Face Geometry Generationcited by 0
  14. 2025Bringing Diversity from Diffusion Models to Semantic-Guided Face Asset Generationcited by 0
  15. 2025Disentangling 3D from Large Vision-Language Models for Controlled Portrait Generationcited by 0
  16. 2025Imagining Alternatives: Towards High-Resolution 3D Counterfactual Medical Image Generation via Language Guidancecited by 0
  17. 2026Dual Diffusion Models for Multi-modal Guided 3D Avatar Generationcited by 0
  18. 2026EmoteGPT: 3D Human Facial Expressions from Natural Language Descriptionscited by 0
  19. 2026RAGMesh with FaME-G2E: Long-Form Text-Driven 3D Face Generation and Editingcited by 0
  20. 2026Long-Tail Adaptive Flow Matching with Explicit Conditional Consistency Guidance for Precise Multimodal Face Synthesiscited by 0