mlsanityData to 14 Oct 2026

You said that?

arXiv 1705.02966 ↗First posted 8 May 2017cs.CVcited by 99

Not on the map. Why?

Abstract (arXiv)

We present a method for generating a video of a talking face. The method takes as inputs: (i) still images of the target face, and (ii) an audio speech segment; and outputs a video of the target face lip synched with the audio. The method runs in real time and is applicable to faces and audio not seen at training time. To achieve this we propose an encoder-decoder CNN model that uses a joint embedding of the face and audio to generate synthesised talking face video frames. The model is trained on tens of hours of unlabelled videos. We also show results of re-dubbing videos using speech from a different person.

Key facts. Cited by 99 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 28; 2022: 24; 2023: 27; 2024: 13; 2025: 5; 2026: 2.

99 papers.

Sort:

By area:

  1. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  2. 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
  3. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  4. 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 209
  5. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 203
  6. 2021MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglementcited by 190
  7. 2021Audio-Driven Emotional Video Portraitscited by 180
  8. 2022EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Modelcited by 151
  9. 2022Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generationcited by 119
  10. 2022VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wildcited by 106
  11. 2023Identity-Preserving Talking Face Generation with Landmark and Appearance Priorscited by 98
  12. 2022Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesiscited by 97
38 more
  1. 2024Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesiscited by 97
  2. 2023StyleTalk: One-shot Talking Head Generation with Controllable Speaking Stylescited by 92
  3. 2021FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learningcited by 90
  4. 2021One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learningcited by 87
  5. 2022Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decompositioncited by 81
  6. 2023Efficient Emotional Adaptation for Audio-Driven Talking-Head Generationcited by 81
  7. 2022Imitator: Personalized Speech-driven 3D Facial Animationcited by 74
  8. 2023DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Videocited by 74
  9. 2021CCVS: Context-aware Controllable Video Synthesiscited by 69
  10. 2021LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalizationcited by 67
  11. 2021Responsive Listening Head Generation: A Benchmark Dataset and Baselinecited by 67
  12. 2021Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templatescited by 58
  13. 2023FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Modelscited by 55
  14. 2021Distilling Audio-Visual Knowledge by Compositional Contrastive Learningcited by 53
  15. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  16. 2024VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesiscited by 50
  17. 2023Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronizationcited by 49
  18. 2023VividTalk: One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Priorcited by 47
  19. 2022Implicit Warping for Animation with Image Setscited by 45
  20. 2022Emotion-Controllable Generalized Talking Face Generationcited by 43
  21. 2021Audio-Visual Synchronisation in the wildcited by 40
  22. 2021Neural Dubber: Dubbing for Videos According to Scriptscited by 39
  23. 2023High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learningcited by 37
  24. 2021Live Speech Portraits: Real-Time Photorealistic Talking-Head Animationcited by 36
  25. 2023TalkCLIP: Talking Head Generation with Text-Guided Expressive Speaking Stylescited by 32
  26. 2021Sparse to Dense Motion Transfer for Face Image Animationcited by 25
  27. 2023DREAM-Talk: Diffusion-based Realistic Emotional Audio-driven Method for Single Image Talking Face Generationcited by 25
  28. 2021Can audio-visual integration strengthen robustness under multimodal attacks?cited by 23
  29. 2022Audio-Driven Talking Face Video Generation with Dynamic Convolution Kernelscited by 23
  30. 2022Learning Visual Styles from Audio-Visual Associationscited by 22
  31. 2022Deep Learning for Visual Speech Analysis: A Surveycited by 21
  32. 2023SynthVSR: Scaling Up Visual Speech Recognition With Synthetic Supervisioncited by 21
  33. 2024Faces that Speak: Jointly Synthesising Talking Face and Speech from Textcited by 21
  34. 2023DaGAN++: Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 20
  35. 2024FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audiocited by 20
  36. 2022Audio-Visual Face Reenactmentcited by 18
  37. 2021AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Personcited by 17
  38. 2021Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesiscited by 16

50 most cited of 99 All 99 (JSON)