G3FA: Geometry-guided GAN for Face Animation
Not on the map. Why?
Abstract (arXiv)
Animating human face images aims to synthesize a desired source identity in a natural-looking way mimicking a driving video's facial movements. In this context, Generative Adversarial Networks have demonstrated remarkable potential in real-time face reenactment using a single source image, yet are constrained by limited geometry consistency compared to graphic-based approaches. In this paper, we introduce Geometry-guided GAN for Face Animation (G3FA) to tackle this limitation. Our novel approach empowers the face animation model to incorporate 3D information using only 2D images, improving the image generation capabilities of the talking head synthesis model. We integrate inverse rendering techniques to extract 3D facial geometry properties, improving the feedback loop to the generator through a weighted average ensemble of discriminators. In our face reenactment model, we leverage 2D motion warping to capture motion dynamics along with orthogonal ray sampling and volume rendering techniques to produce the ultimate visual output. To evaluate the performance of our G3FA, we conducted comprehensive experiments using various evaluation protocols on VoxCeleb2 and TalkingHead benchmarks to demonstrate the effectiveness of our proposed framework compared to the state-of-the-art real-time face animation methods.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 43 of its 52 references that are in the map. First posted on arXiv 23 Aug 2024.
Builds on
43 of 52 references.
References · 43
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Generative Adversarial Networkscited by 5,603
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
- 2017Improved Training of Wasserstein GANscited by 2,223
- 2014Conditional Generative Adversarial Netscited by 2,152
31 more
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
- 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
- 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
- 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
- 2020First Order Motion Model for Image Animationcited by 698
- 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
- 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
- 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
- 2019Few-Shot Adversarial Learning of Realistic Neural Talking Head Modelscited by 284
- 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
- 2021Neural Head Avatars from Monocular RGB Videoscited by 212
- 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
- 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
- 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
- 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
- 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
- 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
- 2023DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animationcited by 136
- 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
- 2020Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatarscited by 116
- 2021Ensembling Off-the-shelf Models for GAN Trainingcited by 84
- 2017Dual Discriminator Generative Adversarial Netscited by 55
- 2023HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Facescited by 48
- 2022Implicit Warping for Animation with Image Setscited by 45
- 2023DPE: Disentanglement of Pose and Expression for General Video Portrait Editingcited by 45
- 2022High-fidelity Facial Avatar Reconstruction from Monocular Video with Generative Priorscited by 28
- 2022NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animationcited by 23
- 2022FNeVR: Neural Volume Rendering for Face Animationcited by 22
- 2021SAFA: Structure Aware Face Animationcited by 20
- 2022Improving 3D-aware Image Synthesis with A Geometry-aware Discriminatorcited by 16
- 2018Dropout-GAN: Learning from a Dynamic Ensemble of Discriminatorscited by 15
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.