mlsanityData to 7 Oct 2026

Semantic 3D-aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field

arXiv 2302.01579 ↗First posted 3 Feb 2023cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Recently 3D-aware GAN methods with neural radiance field have developed rapidly. However, current methods model the whole image as an overall neural radiance field, which limits the partial semantic editability of synthetic results. Since NeRF renders an image pixel by pixel, it is possible to split NeRF in the spatial dimension. We propose a Compositional Neural Radiance Field (CNeRF) for semantic 3D-aware portrait synthesis and manipulation. CNeRF divides the image by semantic regions and learns an independent neural radiance field for each region, and finally fuses them and renders the complete image. Thus we can manipulate the synthesized semantic regions independently, while fixing the other parts unchanged. Furthermore, CNeRF is also designed to decouple shape and texture within each semantic region. Compared to state-of-the-art 3D-aware GAN methods, our approach enables fine-grained semantic region manipulation, while maintaining high-quality 3D-consistent synthesis. The ablation studies show the effectiveness of the structure and loss function used by our method. In addition real image inversion and cartoon portrait 3D editing experiments demonstrate the application potential of our method.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 44 of its 53 references that are in the map. First posted on arXiv 3 Feb 2023.

Builds on

44 of 53 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  6. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  7. 2018Demystifying MMD GANscited by 1,268
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  9. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  10. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  11. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  12. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
32 more
  1. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  2. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  3. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  4. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  5. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  6. 2018Which Training Methods for GANs do actually Converge?cited by 672
  7. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  8. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  9. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  10. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  11. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  12. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  13. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  14. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  15. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  16. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  17. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  18. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  19. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  20. 2020Swapping Autoencoder for Deep Image Manipulationcited by 215
  21. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  22. 2018X2Face: A network for controlling face generation by using images, audio, and pose codescited by 189
  23. 2021FENeRF: Face Editing in Neural Radiance Fieldscited by 112
  24. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  25. 2022Neural Texture Extraction and Distribution for Controllable Person Image Synthesiscited by 55
  26. 2017LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generationcited by 36
  27. 2020RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spacescited by 32
  28. 2019Object Discovery with a Copy-Pasting GANcited by 29
  29. 2022Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fieldscited by 29
  30. 2021Compositional Transformers for Scene Generationcited by 24
  31. 2020Learning to Manipulate Individual Objects in an Imagecited by 22
  32. 2020MUST-GAN: Multi-level Statistics Transfer for Self-driven Person Image Generationcited by 16

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 3; 2025: 1.

10 papers.

  1. 2023AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesiscited by 16
  2. 2023Learning Dense Correspondence for NeRF-Based Face Reenactmentcited by 12
  3. 2023AttriHuman-3D: Editable 3D Human Avatar Generation with Attribute Decomposition and Indexingcited by 8
  4. 2023MaTe3D: Mask-guided Text-based 3D-aware Portrait Editingcited by 3
  5. 2024SemanticHuman-HD: High-Resolution Semantic Disentangled 3D Human Generationcited by 3
  6. 2023A General Implicit Framework for Fast NeRF Composition and Renderingcited by 2
  7. 20243D-SSGAN: Lifting 2D Semantics for 3D-Aware Compositional Portrait Synthesiscited by 2
  8. 2025SuperNeRF-GAN: A Universal 3D-Consistent Super-Resolution Framework for Efficient and Enhanced 3D-Aware Image Synthesiscited by 2
  9. 2023Freestyle 3D-Aware Portrait Synthesis Based on Compositional Generative Priorscited by 0
  10. 2024Pomo3D: 3D-Aware Portrait Accessorizing and Morecited by 0