mlsanityData to 7 Oct 2026

3D-Aware Semantic-Guided Generative Model for Human Synthesis

arXiv 2112.01422 ↗First posted 2 Dec 2021cs.CVcited by 31

Not on the map. Why?

Abstract (arXiv)

Generative Neural Radiance Field (GNeRF) models, which extract implicit 3D representations from 2D images, have recently been shown to produce realistic images representing rigid/semi-rigid objects, such as human faces or cars. However, they usually struggle to generate high-quality images representing non-rigid objects, such as the human body, which is of a great interest for many computer graphics applications. This paper proposes a 3D-aware Semantic-Guided Generative Model (3D-SGAN) for human image synthesis, which combines a GNeRF with a texture generator. The former learns an implicit 3D representation of the human body and outputs a set of 2D semantic segmentation masks. The latter transforms these semantic masks into a real image, adding a realistic texture to the human appearance. Without requiring additional 3D information, our model can learn 3D human representations with a photo-realistic, controllable generation. Our experiments on the DeepFashion dataset show that 3D-SGAN significantly outperforms the most recent baselines. The code is available at https://github.com/zhangqianhui/3DSGAN

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 74 of its 89 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

74 of 89 references.

References · 74

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2017Improved Training of Wasserstein GANscited by 2,223
  11. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  12. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
38 more
  1. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  2. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  3. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  4. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  5. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  6. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  7. 2020First Order Motion Model for Image Animationcited by 698
  8. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  9. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  10. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  11. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  12. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  13. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  14. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  15. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  16. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  17. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  18. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  19. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  20. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  21. 2017VITON: An Image-based Virtual Try-on Networkcited by 285
  22. 2017Pose Guided Person Image Generationcited by 251
  23. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  24. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  25. 2020Controllable Person Image Synthesis with Attribute-Decomposed GANcited by 150
  26. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  27. 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
  28. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  29. 2020Patch2Pix: Epipolar-Guided Pixel-Level Correspondencescited by 128
  30. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  31. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  32. 2021FENeRF: Face Editing in Neural Radiance Fieldscited by 112
  33. 2019Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesiscited by 104
  34. 2020Deep Image Spatial Transformation for Person Image Generationcited by 100
  35. 2017Disentangled Person Image Generationcited by 98
  36. 2018Synthesizing Images of Humans in Unseen Posescited by 96
  37. 2018Dense Pose Transfercited by 83
  38. 2020XingGAN for Person Image Generationcited by 81

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 15; 2023: 9; 2024: 4; 2025: 2; 2026: 1.

31 papers.

Sort:

By area:

  1. 2022VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Gridscited by 133
  2. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  3. 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
  4. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  5. 2022AvatarGen: A 3D Generative Model for Animatable Human Avatarscited by 59
  6. 2022AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatarscited by 55
  7. 2022HumanGen: Generating Human Radiance Fields with Explicit Priorscited by 33
  8. 2023Disentangled Clothed Avatar Generation from Text Descriptionscited by 33
  9. 2024Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Imagecited by 32
  10. 2022Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fieldscited by 29
  11. 20233D-aware Conditional Image Synthesiscited by 24
  12. 2022A Survey on Deep Generative 3D-aware Image Synthesiscited by 16
19 more
  1. 2023Efficient 3D Articulated Human Generation with Layered Surface Volumescited by 13
  2. 2022Human Image Generation: A Comprehensive Surveycited by 11
  3. 2022Bipartite Graph Reasoning GANs for Person Pose and Facial Image Synthesiscited by 9
  4. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  5. 2022Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generationcited by 8
  6. 2023VeRi3D: Generative Vertex-based Radiance Fields for 3D Controllable Human Image Synthesiscited by 8
  7. 2023WildFusion: Learning 3D-Aware Latent Diffusion Models in View Spacecited by 8
  8. 2023AttriHuman-3D: Editable 3D Human Avatar Generation with Attribute Decomposition and Indexingcited by 8
  9. 2023Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Modelscited by 6
  10. 2023MVHuman: Tailoring 2D Diffusion with Multi-view Sampling For Realistic 3D Human Generationcited by 6
  11. 2022Unsupervised High-Resolution Portrait Gaze Correction and Animationcited by 5
  12. 2024SemanticHuman-HD: High-Resolution Semantic Disentangled 3D Human Generationcited by 3
  13. 2025Enhanced Multi-Scale Cross-Attention for Person Image Generationcited by 3
  14. 20243D-aware Image Generation and Editing with Multi-modal Conditionscited by 2
  15. 2025PARTE: Part-Guided Texturing for 3D Human Reconstruction from a Single Imagecited by 2
  16. 2022Deep Generative Models on 3D Representations: A Surveycited by 1
  17. 2023Partial-View Object View Synthesis via Filtered Inversioncited by 1
  18. 2022AvatarGen: a 3D Generative Model for Animatable Human Avatarscited by 0
  19. 2026High-Fidelity 3D Facial Avatar Synthesis with Controllable Fine-Grained Expressionscited by 0