mlsanityData to 7 Oct 2026

Fast Text-to-3D-Aware Face Generation and Manipulation via Direct Cross-modal Mapping and Geometric Regularization

arXiv 2403.06702 ↗First posted 11 Mar 2024cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Text-to-3D-aware face (T3D Face) generation and manipulation is an emerging research hot spot in machine learning, which still suffers from low efficiency and poor quality. In this paper, we propose an End-to-End Efficient and Effective network for fast and accurate T3D face generation and manipulation, termed E3-FaceNet. Different from existing complex generation paradigms, E3-FaceNet resorts to a direct mapping from text instructions to 3D-aware visual space. We introduce a novel Style Code Enhancer to enhance cross-modal semantic alignment, alongside an innovative Geometric Regularization objective to maintain consistency across multi-view generations. Extensive experiments on three benchmark datasets demonstrate that E3-FaceNet can not only achieve picture-like 3D face generation and manipulation, but also improve inference speed by orders of magnitudes. For instance, compared with Latent3D, E3-FaceNet speeds up the five-view generations by almost 470 times, while still exceeding in generation quality. Our code is released at https://github.com/Aria-Zhangjl/E3-FaceNet.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

42 of 60 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  9. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  10. 2014Conditional Generative Adversarial Netscited by 2,152
  11. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  12. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
30 more
  1. 2018Demystifying MMD GANscited by 1,268
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  3. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  4. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  5. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  6. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  7. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  8. 2018Which Training Methods for GANs do actually Converge?cited by 672
  9. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  10. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  11. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  12. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  13. 2021GAN Inversion: A Surveycited by 366
  14. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  15. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  16. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  17. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  18. 2023DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidancecited by 79
  19. 2022ClipFace: Text-guided Editing of Textured 3D Morphable Modelscited by 59
  20. 2021ClipMatrix: Text-controlled Creation of 3D Textured Meshescited by 40
  21. 2022Text and Image Guided 3D Avatar Generation and Manipulationcited by 35
  22. 2019Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networkscited by 34
  23. 2022AnyFace: Free-style Text-to-Face Synthesis and Manipulationcited by 33
  24. 2019A Real-time Global Inference Network for One-stage Referring Expression Comprehensioncited by 32
  25. 2023High-Fidelity 3D Face Generation from Natural Language Descriptionscited by 32
  26. 2022Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Taskscited by 21
  27. 2022A Survey on Deep Generative 3D-aware Image Synthesiscited by 16
  28. 2023Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Imagescited by 12
  29. 20233D-CLFusion: Fast Text-to-3D Rendering with Contrastive Latent Diffusioncited by 4
  30. 2023Text2Face: A Multi-Modal 3D Face Modelcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 3.

8 papers.

  1. 2024FlashSloth: Lightning Multimodal Large Language Models via Embedded Visual Compressioncited by 10
  2. 2025AdaFlow: Efficient Long Video Editing via Adaptive Attention Slimming And Keyframe Selectioncited by 5
  3. 2025Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generationcited by 4
  4. 2024Revealing Directions for Text-guided 3D Face Editingcited by 2
  5. 2025Multivariate Diffusion Transformer with Decoupled Attention for High-Fidelity Mask-Text Collaborative Facial Generationcited by 1
  6. 2026CustomTex: High-fidelity Indoor Scene Texturing via Multi-Reference Customizationcited by 1
  7. 2026Detangled: A Framework for Creating, Editing, and Inferencing Feature Rich Hair Strandscited by 0
  8. 2026Long-Tail Adaptive Flow Matching with Explicit Conditional Consistency Guidance for Precise Multimodal Face Synthesiscited by 0