mlsanityData to 7 Oct 2026

One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing

arXiv 2011.15126 ↗First posted 30 Nov 2020cs.CVcited by 438

Not on the map. Why?

Cited by 58 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a neural talking-head video synthesis model and demonstrate its application to video conferencing. Our model learns to synthesize a talking-head video using a source image containing the target person's appearance and a driving video that dictates the motion in the output. Our motion is encoded based on a novel keypoint representation, where the identity-specific and motion-related information is decomposed unsupervisedly. Extensive experimental validation shows that our model outperforms competing methods on benchmark datasets. Moreover, our compact keypoint representation enables a video conferencing system that achieves the same visual quality as the commercial H.264 standard while only using one-tenth of the bandwidth. Besides, we show our keypoint representation allows the user to rotate the head during synthesis, which is useful for simulating face-to-face video conferencing experiences.

Key facts. Cited by 438 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 71; 2023: 88; 2024: 108; 2025: 102; 2026: 41.

438 papers.

Sort:

By area:

  1. 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 380
  2. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  3. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  4. 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
  5. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 271
  6. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
  7. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  8. 2022Thin-Plate Spline Motion Model for Image Animationcited by 209
  9. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
  10. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
  11. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 198
  12. 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
38 more
  1. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  2. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
  3. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 172
  4. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  5. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  6. 2022EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Modelcited by 151
  7. 2023DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animationcited by 136
  8. 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
  9. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
  10. 2024DF40: Toward Next-Generation Deepfake Detectioncited by 126
  11. 2022Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generationcited by 119
  12. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  13. 2022Generating Long Videos of Dynamic Scenescited by 113
  14. 2022VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wildcited by 106
  15. 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
  16. 2022Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesiscited by 97
  17. 2023Identity-Preserving Talking Face Generation with Landmark and Appearance Priorscited by 97
  18. 2024Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesiscited by 97
  19. 2024Generalizable and Animatable Gaussian Head Avatarcited by 95
  20. 2022Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesiscited by 93
  21. 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attentioncited by 88
  22. 2024V-Express: Conditional Dropout for Progressive Training of Portrait Video Generationcited by 85
  23. 2023Efficient Emotional Adaptation for Audio-Driven Talking-Head Generationcited by 81
  24. 2024EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesiscited by 79
  25. 2024GPAvatar: Generalizable and Precise Head Avatar from Image(s)cited by 74
  26. 2022AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxelscited by 73
  27. 2023DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Videocited by 73
  28. 2022MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptationcited by 69
  29. 2023One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Fieldcited by 68
  30. 2021Responsive Listening Head Generation: A Benchmark Dataset and Baselinecited by 67
  31. 2022Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptationcited by 67
  32. 2022Fine-Grained Face Swapping via Regional GAN Inversioncited by 67
  33. 2024AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encodingcited by 67
  34. 2023A Comprehensive Review of Data-Driven Co-Speech Gesture Generationcited by 66
  35. 2024Deepfake Generation and Detection: A Benchmark and Surveycited by 66
  36. 2024Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Controlcited by 66
  37. 2022SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memorycited by 65
  38. 2024EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatarscited by 65

50 most cited of 438 All 438 (JSON)