mlsanityData to 7 Oct 2026

Few-Shot Adversarial Learning of Realistic Neural Talking Head Models

arXiv 1905.08233 ↗First posted 20 May 2019cs.CV cs.GR cs.LGcited by 284

Not on the map. Why?

Abstract (arXiv)

Several recent works have shown how highly realistic human head images can be obtained by training convolutional neural networks to generate them. In order to create a personalized talking head model, these works require training on a large dataset of images of a single person. However, in many practical scenarios, such personalized talking head models need to be learned from a few image views of a person, potentially even a single image. Here, we present a system with such few-shot capability. It performs lengthy meta-learning on a large dataset of videos, and after that is able to frame few- and one-shot learning of neural talking head models of previously unseen people as adversarial training problems with high capacity generators and discriminators. Crucially, the system is able to initialize the parameters of both the generator and the discriminator in a person-specific way, so that training can be based on just a few images and done quickly, despite the need to tune tens of millions of parameters. We show that such an approach is able to learn highly realistic and personalized talking head models of new people and even portrait paintings.

Key facts. Cited by 284 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 74; 2022: 66; 2023: 70; 2024: 36; 2025: 24; 2026: 14.

284 papers.

Sort:

By area:

  1. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  2. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  3. 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
  4. 2021Motion Representations for Articulated Animationcited by 253
  5. 2022Thin-Plate Spline Motion Model for Image Animationcited by 209
  6. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
  7. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  8. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
  9. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  10. 2023EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animationcited by 141
  11. 2023DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animationcited by 136
  12. 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
38 more
  1. 2022Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generationcited by 119
  2. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  3. 2021Image-to-Image Translation: Methods and Applicationscited by 116
  4. 2023Emotional Speech-Driven Animation with Content-Emotion Disentanglementcited by 98
  5. 2022Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesiscited by 97
  6. 2024Generalizable and Animatable Gaussian Head Avatarcited by 95
  7. 2021FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learningcited by 90
  8. 2023StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generatorcited by 89
  9. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  10. 2022DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Renderingcited by 80
  11. 2021Deepfake Video Detection Using Convolutional Vision Transformercited by 79
  12. 2024EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesiscited by 79
  13. 2021MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Imagescited by 78
  14. 2022AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxelscited by 73
  15. 2021Region-aware Adaptive Instance Normalization for Image Harmonizationcited by 71
  16. 2022MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptationcited by 69
  17. 2023BlendFace: Re-designing Identity Encoders for Face-Swappingcited by 66
  18. 2024Deepfake Generation and Detection: A Benchmark and Surveycited by 66
  19. 2024EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatarscited by 65
  20. 2024Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizercited by 64
  21. 2021StylePeople: A Generative Model of Fullbody Human Avatarscited by 62
  22. 2022Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Productioncited by 61
  23. 2023Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generationcited by 61
  24. 2024AVFF: Audio-Visual Feature Fusion for Video Deepfake Detectioncited by 59
  25. 2021Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowdscited by 58
  26. 2021Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templatescited by 57
  27. 2022Talking Head Generation with Probabilistic Audio-to-Visual Diffusion Priorscited by 55
  28. 2022Audio-Driven Co-Speech Gesture Video Generationcited by 54
  29. 20223D GAN Inversion for Controllable Portrait Image Animationcited by 53
  30. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  31. 2023Multimodal Automated Fact-Checking: A Surveycited by 51
  32. 2021OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wildcited by 50
  33. 2023LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatarcited by 50
  34. 2022Deepfake Detection: A Comprehensive Survey from the Reliability Perspectivecited by 48
  35. 2023Face Animation with an Attribute-Guided Diffusion Modelcited by 48
  36. 2023HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Facescited by 48
  37. 2023VividTalk: One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Priorcited by 47
  38. 2021Write-a-speaker: Text-based Emotional and Rhythmic Talking-head Generationcited by 46

50 most cited of 284 All 284 (JSON)