mlsanityData to 7 Oct 2026

AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections

arXiv 2309.02186 ↗First posted 5 Sep 2023cs.CV cs.AI cs.GRcited by 16

Not on the map. Why?

Abstract (arXiv)

Previous animatable 3D-aware GANs for human generation have primarily focused on either the human head or full body. However, head-only videos are relatively uncommon in real life, and full body generation typically does not deal with facial expression control and still has challenges in generating high-quality results. Towards applicable video avatars, we present an animatable 3D-aware GAN that generates portrait images with controllable facial expression, head pose, and shoulder movements. It is a generative model trained on unstructured 2D image collections without using 3D or video data. For the new task, we base our method on the generative radiance manifold representation and equip it with learnable facial and head-shoulder deformations. A dual-camera rendering and adversarial learning scheme is proposed to improve the quality of the generated faces, which is critical for portrait images. A pose deformation processing network is developed to generate plausible deformations for challenging regions such as long hair. Experiments show that our method, trained on unstructured 2D images, can generate diverse and high-quality 3D portraits with desired control over different properties.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 56 of its 70 references that are in the map. First posted on arXiv 5 Sep 2023.

Builds on

56 of 70 references.

References · 56

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2018Demystifying MMD GANscited by 1,268
  4. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  5. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  6. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  7. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2020First Order Motion Model for Image Animationcited by 698
  10. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  11. 2018Which Training Methods for GANs do actually Converge?cited by 672
  12. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
38 more
  1. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  2. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  3. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  4. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  5. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  6. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 347
  7. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  8. 2019Few-Shot Adversarial Learning of Realistic Neural Talking Head Modelscited by 284
  9. 2018Deep Video Portraitscited by 265
  10. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  11. 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 219
  12. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  13. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
  14. 2018X2Face: A network for controlling face generation by using images, audio, and pose codescited by 189
  15. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  16. 2021Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Renderingcited by 158
  17. 2022StyleGAN-Human: A Data-Centric Odyssey of Human Generationcited by 143
  18. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  19. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  20. 2020Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondencecited by 119
  21. 2020Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimationcited by 113
  22. 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
  23. 2020LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registrationcited by 108
  24. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  25. 2020HeadGAN: One-shot Neural Head Synthesis and Editingcited by 105
  26. 2019Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesiscited by 104
  27. 2022Generative Neural Articulated Radiance Fieldscited by 101
  28. 2022GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifoldscited by 73
  29. 2022gDNA: Towards Generative Detailed Neural Avatarscited by 72
  30. 2022Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representationscited by 65
  31. 2020Lifting 2D StyleGAN for 3D-Aware Face Generationcited by 64
  32. 2022AvatarGen: A 3D Generative Model for Animatable Human Avatarscited by 59
  33. 2022Generative Multiplane Images: Making a 2D GAN 3D-Awarecited by 57
  34. 2023AG3D: Learning to Generate 3D Avatars from 2D Image Collectionscited by 55
  35. 2022MPS-NeRF: Generalizable 3D Human Rendering from Multiview Imagescited by 52
  36. 20233D generation on ImageNetcited by 47
  37. 2022AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatarscited by 46
  38. 2022InsetGAN for Full-Body Image Generationcited by 45

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 5; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
  2. 2023Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Datacited by 55
  3. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  4. 2024StereoDiffusion: Training-Free Stereo Image Generation Using Latent Diffusion Modelscited by 24
  5. 2024Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Priorcited by 15
  6. 2024Portrait Video Editing Empowered by Multimodal Generative Priorscited by 12
  7. 2024GPHM: Gaussian Parametric Head Model for Monocular Head Avatar Reconstructioncited by 10
  8. 2025Human-Centric Foundation Models: Perception, Generation and Agentic Modelingcited by 10
  9. 2024NeRFFaceSpeech: One-shot Audio-driven 3D Talking Head Synthesis via Generative Priorcited by 8
  10. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Datacited by 7
  11. 2025LMME3DHF: Benchmarking and Evaluating Multimodal 3D Human Face Generation with LMMscited by 7
  12. 2024Free-viewpoint Human Animation with Pose-correlated Reference Selectioncited by 5
4 more
  1. 2024Generating Editable Head Avatars with 3D Gaussian GANscited by 5
  2. 2025Controlling Avatar Diffusion with Learnable Gaussian Embeddingcited by 3
  3. 2025Bringing Your Portrait to 3D Presencecited by 3
  4. 2026Instruction-Driven 3D Facial Expression Generation and Transitioncited by 1