mlsanityData to 14 Oct 2026

IDE-3D: Interactive Disentangled Editing for High-Resolution 3D-aware Portrait Synthesis

arXiv 2205.15517 ↗First posted 31 May 2022cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Existing 3D-aware facial generation methods face a dilemma in quality versus editability: they either generate editable results in low resolution or high-quality ones with no editing flexibility. In this work, we propose a new approach that brings the best of both worlds together. Our system consists of three major components: (1) a 3D-semantics-aware generative model that produces view-consistent, disentangled face images and semantic masks; (2) a hybrid GAN inversion approach that initialize the latent codes from the semantic and texture encoder, and further optimized them for faithful reconstruction; and (3) a canonical editor that enables efficient manipulation of semantic masks in canonical view and product high-quality editing results. Our approach is competent for many applications, e.g. free-view face drawing, editing, and style control. Both quantitative and qualitative results show that our method reaches the state-of-the-art in terms of photorealism, faithfulness, and efficiency.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 52 of its 68 references that are in the map. First posted on arXiv 31 May 2022.

Builds on

52 of 68 references.

References · 52

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  3. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  4. 2018Demystifying MMD GANscited by 1,293
  5. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  6. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  7. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  8. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  9. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  10. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  11. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  12. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 695
38 more
  1. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  2. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  3. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
  4. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  5. 2020GANSpace: Discovering Interpretable GAN Controlscited by 614
  6. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 482
  7. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
  8. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  9. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  10. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  11. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 394
  12. 2021GAN Inversion: A Surveycited by 367
  13. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  14. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 349
  15. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  16. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 268
  17. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  18. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  19. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  20. 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 220
  21. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  22. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
  23. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  24. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  25. 2021StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generatorscited by 164
  26. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  27. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  28. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  29. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  30. 2021FENeRF: Face Editing in Neural Radiance Fieldscited by 112
  31. 20213D-aware Image Synthesis via Learning Structural and Textural Representationscited by 109
  32. 2022FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Datasetcited by 98
  33. 2022GIRAFFE HD: A High-Resolution 3D-aware Generative Modelcited by 86
  34. 2022Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translationcited by 73
  35. 2020SofGAN: A Portrait Image Generator with Dynamic Stylingcited by 65
  36. 2022Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representationscited by 65
  37. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  38. 20223D GAN Inversion for Controllable Portrait Image Animationcited by 53

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3.

5 papers.

  1. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  2. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  3. 2023SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Fieldcited by 94
  4. 20223D GAN Inversion with Facial Symmetry Priorcited by 39
  5. 20233D Face Arbitrary Style Transfercited by 0