mlsanityData to 7 Oct 2026

3D-SSGAN: Lifting 2D Semantics for 3D-Aware Compositional Portrait Synthesis

arXiv 2401.03764 ↗First posted 8 Jan 2024cs.CV cs.GRcited by 2

Not on the map. Why?

Abstract (arXiv)

Existing 3D-aware portrait synthesis methods can generate impressive high-quality images while preserving strong 3D consistency. However, most of them cannot support the fine-grained part-level control over synthesized images. Conversely, some GAN-based 2D portrait synthesis methods can achieve clear disentanglement of facial regions, but they cannot preserve view consistency due to a lack of 3D modeling abilities. To address these issues, we propose 3D-SSGAN, a novel framework for 3D-aware compositional portrait image synthesis. First, a simple yet effective depth-guided 2D-to-3D lifting module maps the generated 2D part features and semantics to 3D. Then, a volume renderer with a novel 3D-aware semantic mask renderer is utilized to produce the composed face features and corresponding masks. The whole framework is trained end-to-end by discriminating between real and synthesized 2D images and their semantic masks. Quantitative and qualitative evaluations demonstrate the superiority of 3D-SSGAN in controllable part-level synthesis while preserving 3D view consistency.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 8 Jan 2024.

Builds on

36 of 45 references.

References · 36

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  3. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  4. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  5. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  6. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  7. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  8. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  9. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  10. 2021Advances in Neural Renderingcited by 365
  11. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  12. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
24 more
  1. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  2. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  3. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  4. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  5. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  6. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  7. 2021FENeRF: Face Editing in Neural Radiance Fieldscited by 112
  8. 20213D-aware Image Synthesis via Learning Structural and Textural Representationscited by 109
  9. 2021Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editingcited by 91
  10. 2022GIRAFFE HD: A High-Resolution 3D-aware Generative Modelcited by 86
  11. 2022GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifoldscited by 73
  12. 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
  13. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  14. 2017LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generationcited by 36
  15. 2022TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editingcited by 36
  16. 2021Diagonal Attention and Style-based GAN for Content-Style Disentanglement in Image Generation and Translationcited by 35
  17. 2022State-of-the-Art in the Architecture, Methods and Applications of StyleGANcited by 34
  18. 2022NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fieldscited by 33
  19. 2020RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spacescited by 32
  20. 2022Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fieldscited by 29
  21. 2021StyleFusion: A Generative Model for Disentangling Spatial Segmentscited by 28
  22. 2023Semantic 3D-aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Fieldcited by 10
  23. 2022Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generationcited by 8
  24. 2021SAC-GAN: Structure-Aware Image Compositioncited by 5

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024SemanticHuman-HD: High-Resolution Semantic Disentangled 3D Human Generationcited by 3
  2. 2025SuperNeRF-GAN: A Universal 3D-Consistent Super-Resolution Framework for Efficient and Enhanced 3D-Aware Image Synthesiscited by 2