mlsanityData to 7 Oct 2026

3D GAN Inversion for Controllable Portrait Image Animation

arXiv 2203.13441 ↗First posted 25 Mar 2022cs.CV cs.GR cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

Millions of images of human faces are captured every single day; but these photographs portray the likeness of an individual with a fixed pose, expression, and appearance. Portrait image animation enables the post-capture adjustment of these attributes from a single image while maintaining a photorealistic reconstruction of the subject's likeness or identity. Still, current methods for portrait image animation are typically based on 2D warping operations or manipulations of a 2D generative adversarial network (GAN) and lack explicit mechanisms to enforce multi-view consistency. Thus these methods may significantly alter the identity of the subject, especially when the viewpoint relative to the camera is changed. In this work, we leverage newly developed 3D GANs, which allow explicit control over the pose of the image subject with multi-view consistency. We propose a supervision strategy to flexibly manipulate expressions with 3D morphable models, and we show that the proposed method also supports editing appearance attributes, such as age or hairstyle, by interpolating within the latent space of the GAN. The proposed technique for portrait image animation outperforms previous methods in terms of image quality, identity preservation, and pose transfer while also supporting attribute editing.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 64 of its 82 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

64 of 82 references.

References · 64

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  8. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  9. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  10. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  11. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  12. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
38 more
  1. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  2. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  3. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  4. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  5. 2020First Order Motion Model for Image Animationcited by 698
  6. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  7. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  8. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  9. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  10. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  11. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  12. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  13. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  14. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  15. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  16. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  17. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  18. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  19. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  20. 2021GAN Inversion: A Surveycited by 366
  21. 2021Advances in Neural Renderingcited by 365
  22. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  23. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  24. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  25. 2019Few-Shot Adversarial Learning of Realistic Neural Talking Head Modelscited by 284
  26. 2020State of the Art on Neural Renderingcited by 276
  27. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 267
  28. 2018Deep Video Portraitscited by 265
  29. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  30. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
  31. 2019Few-shot Video-to-Video Synthesiscited by 196
  32. 2018X2Face: A network for controlling face generation by using images, audio, and pose codescited by 189
  33. 2020Editing in Style: Uncovering the Local Semantics of GANscited by 183
  34. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  35. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  36. 2018Deep Appearance Models for Face Renderingcited by 164
  37. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  38. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 15; 2023: 22; 2024: 7; 2025: 8; 2026: 1.

53 papers.

Sort:

By area:

  1. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  2. 2022Generative Neural Articulated Radiance Fieldscited by 101
  3. 2022Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesiscited by 93
  4. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  5. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  6. 2024Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizercited by 64
  7. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  8. 2023Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Datacited by 55
  9. 20223D GAN Inversion with Pose Optimizationcited by 44
  10. 2023DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatarscited by 42
  11. 2022NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fieldscited by 33
  12. 2023Control3Diff: Learning Controllable 3D Diffusion Models from Single-view Imagescited by 27
38 more
  1. 2023DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesiscited by 26
  2. 2022Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversioncited by 24
  3. 2022NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animationcited by 23
  4. 2023LPFF: A Portrait Dataset for Face Generators Across Large Posescited by 22
  5. 2023InvertAvatar: Incremental GAN Inversion for Generalized Head Avatarscited by 19
  6. 20223D-Aware Video Generationcited by 17
  7. 2022A Survey on Deep Generative 3D-aware Image Synthesiscited by 16
  8. 2023VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANscited by 16
  9. 2023Single-Shot Implicit Morphable Faces with Consistent Texture Parameterizationcited by 16
  10. 2025AvatarArtist: Open-Domain 4D Avatarizationcited by 15
  11. 2023GAN-Avatar: Controllable Personalized GAN-based Human Head Avatarcited by 14
  12. 2023Learning Dense Correspondence for NeRF-Based Face Reenactmentcited by 12
  13. 2022Score-Guided Intermediate Layer Optimization: Fast Langevin Mixing for Inverse Problemscited by 11
  14. 2023PREIM3D: 3D Consistent Precise Image Attribute Editing from a Single Imagecited by 11
  15. 2023InstructPix2NeRF: Instructed 3D Portrait Editing from a Single Imagecited by 10
  16. 2024Real-time 3D-aware Portrait Editing from a Single Imagecited by 10
  17. 2025Monocular and Generalizable Gaussian Talking Head Animationcited by 10
  18. 2022Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generationcited by 8
  19. 20233DPortraitGAN: Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Posescited by 8
  20. 2022Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis from Monocular Imagecited by 7
  21. 2023DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANscited by 6
  22. 2022IDE-3D: Interactive Disentangled Editing for High-Resolution 3D-aware Portrait Synthesiscited by 5
  23. 2022LatentSwap3D: Semantic Edits on 3D Image GANscited by 5
  24. 2023In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editingcited by 4
  25. 2023TEGLO: High Fidelity Canonical Texture Mapping from Single-View Imagescited by 4
  26. 2023MaskRenderer: 3D-Infused Multi-Mask Realistic Face Reenactmentcited by 3
  27. 2024Coherent 3D Portrait Video Reconstruction via Triplane Fusioncited by 3
  28. 2024Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Priorcited by 2
  29. 2025Total-Editing: Head Avatar with Editable Appearance, Motion, and Lightingcited by 2
  30. 2024DiffDesign: Controllable Diffusion with Meta Prior for Efficient Interior Design Generationcited by 1
  31. 2025Follow Your Motion: A Generic Temporal Consistency Portrait Editing Framework with Trajectory Guidancecited by 1
  32. 2025FactorPortrait: Controllable Portrait Animation via Disentangled Expression, Pose, and Viewpointcited by 1
  33. 2026From Blurry to Believable: Enhancing Low-quality Talking Heads with 3D Generative Priorscited by 1
  34. 2023ZIGNeRF: Zero-shot 3D Scene Representation with Invertible Generative Neural Radiance Fieldscited by 0
  35. 2023Freestyle 3D-Aware Portrait Synthesis Based on Compositional Generative Priorscited by 0
  36. 2023Facial Reenactment Through a Personalized Generatorcited by 0
  37. 2024Efficient 3D-Aware Facial Image Editing via Attribute-Specific Prompt Learningcited by 0
  38. 2024Coherent3D: Coherent 3D Portrait Video Reconstruction via Triplane Fusioncited by 0

50 most cited of 53 All 53 (JSON)