mlsanityData to 14 Oct 2026

Free-HeadGAN: Neural Talking Head Synthesis with Explicit Gaze Control

arXiv 2208.02210 ↗First posted 3 Aug 2022cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

We present Free-HeadGAN, a person-generic neural talking head synthesis system. We show that modeling faces with sparse 3D facial landmarks are sufficient for achieving state-of-the-art generative performance, without relying on strong statistical priors of the face, such as 3D Morphable Models. Apart from 3D pose and facial expressions, our method is capable of fully transferring the eye gaze, from a driving actor to a source identity. Our complete pipeline consists of three components: a canonical 3D key-point estimator that regresses 3D pose and expression-related deformations, a gaze estimation network and a generator that is built upon the architecture of HeadGAN. We further experiment with an extension of our generator to accommodate few-shot learning using an attention mechanism, in case more than one source images are available. Compared to the latest models for reenactment and motion transfer, our system achieves higher photo-realism combined with superior identity preservation, while offering explicit gaze control.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 56 of its 80 references that are in the map. First posted on arXiv 3 Aug 2022.

Builds on

56 of 80 references.

References · 56

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  5. 2014Conditional Generative Adversarial Netscited by 2,162
  6. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  7. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  8. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  9. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  10. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,204
  11. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
  12. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
38 more
  1. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  2. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  3. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
  4. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  5. 2020First Order Motion Model for Image Animationcited by 704
  6. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 632
  7. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  8. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 444
  9. 2018Video-to-Video Synthesiscited by 390
  10. 2015WIDER FACE: A Face Detection Benchmarkcited by 298
  11. 2019Few-Shot Adversarial Learning of Realistic Neural Talking Head Modelscited by 284
  12. 2018Deep Video Portraitscited by 267
  13. 2021Motion Representations for Articulated Animationcited by 255
  14. 2019Neural Voice Puppetry: Audio-driven Facial Reenactmentcited by 250
  15. 2017Geometric GANcited by 233
  16. 2018Animating Arbitrary Objects via Deep Motion Transfercited by 208
  17. 2019Few-shot Video-to-Video Synthesiscited by 197
  18. 2018X2Face: A network for controlling face generation by using images, audio, and pose codescited by 189
  19. 2019Gaze360: Physically Unconstrained Gaze Estimation in the Wildcited by 174
  20. 2016Eye Tracking for Everyonecited by 155
  21. 2015Appearance-Based Gaze Estimation in the Wildcited by 153
  22. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  23. 2019Realistic Speech-Driven Facial Animation with GANscited by 142
  24. 2020ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variationcited by 141
  25. 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
  26. 2020Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatarscited by 118
  27. 2020HeadGAN: One-shot Neural Head Synthesis and Editingcited by 105
  28. 2019MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targetscited by 90
  29. 2019Few-Shot Adaptive Gaze Estimationcited by 77
  30. 2018Deep Pictorial Gaze Estimationcited by 49
  31. 2016Automatic Face Reenactmentcited by 43
  32. 2019Improving Few-Shot User-Specific Gaze Adaptation via Gaze Redirection Synthesiscited by 43
  33. 2019Unsupervised Representation Learning for Gaze Estimationcited by 42
  34. 2021Generalizing Gaze Estimation with Outlier-guided Collaborative Adaptationcited by 42
  35. 2020Head2Head: Video-based Neural Head Synthesiscited by 40
  36. 2018Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settingscited by 38
  37. 2020Head2Head++: Deep Facial Attributes Re-Targetingcited by 30
  38. 2020Motion-supervised Co-Part Segmentationcited by 28

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 2; 2025: 2; 2026: 1.

10 papers.

  1. 2023Identity-Preserving Talking Face Generation with Landmark and Appearance Priorscited by 98
  2. 2023Efficient Emotional Adaptation for Audio-Driven Talking-Head Generationcited by 81
  3. 2024Deepfake Generation and Detection: A Benchmark and Surveycited by 67
  4. 2023Interactive Conversational Head Generationcited by 14
  5. 2024A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editingcited by 5
  6. 2023One-Shot Face Video Re-enactment using Hybrid Latent Spaces of StyleGAN2cited by 3
  7. 2023FIANCEE: Faster Inference of Adversarial Networks via Conditional Early Exitscited by 3
  8. 2026FG-Portrait: 3D Flow Guided Editable Portrait Animationcited by 1
  9. 2025STSA: Spatial-Temporal Semantic Alignment for Visual Dubbingcited by 0
  10. 2025JOLT3D: Joint Learning of Talking Heads and 3DMM Parameters with Application to Lip-Synccited by 0