mlsanityData to 7 Oct 2026

Text-Driven Diverse Facial Texture Generation via Progressive Latent-Space Refinement

arXiv 2404.09540 ↗First posted 15 Apr 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Automatic 3D facial texture generation has gained significant interest recently. Existing approaches may not support the traditional physically based rendering pipeline or rely on 3D data captured by Light Stage. Our key contribution is a progressive latent space refinement approach that can bootstrap from 3D Morphable Models (3DMMs)-based texture maps generated from facial images to generate high-quality and diverse PBR textures, including albedo, normal, and roughness. It starts with enhancing Generative Adversarial Networks (GANs) for text-guided and diverse texture generation. To this end, we design a self-supervised paradigm to overcome the reliance on ground truth 3D textures and train the generative model with only entangled texture maps. Besides, we foster mutual enhancement between GANs and Score Distillation Sampling (SDS). SDS boosts GANs with more generative modes, while GANs promote more efficient optimization of SDS. Furthermore, we introduce an edge-aware SDS for multi-view consistent facial structure. Experiments demonstrate that our method outperforms existing 3D texture generation methods regarding photo-realistic quality, diversity, and efficiency.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 65 of its 73 references that are in the map. First posted on arXiv 15 Apr 2024.

Builds on

65 of 73 references.

References · 65

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  10. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  11. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  12. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
38 more
  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  2. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  3. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  4. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  5. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  6. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  7. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  11. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  12. 2015Holistically-Nested Edge Detectioncited by 590
  13. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  14. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  15. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  16. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  17. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  18. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  19. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  20. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  21. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  22. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  23. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  24. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  25. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  26. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  27. 2023TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessmentcited by 220
  28. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  29. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  30. 2022Diffusion-GAN: Training GANs with Diffusioncited by 203
  31. 2018Nonlinear 3D Face Morphable Modelcited by 157
  32. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  33. 2022VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Gridscited by 133
  34. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  35. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  36. 20213D-aware Image Synthesis via Learning Structural and Textural Representationscited by 109
  37. 2019Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesiscited by 104
  38. 2020Learning Formation of Physically-Based Face Attributescited by 94

50 most cited of 65 All 65 (JSON)

Built on by

Not cited yet.