mlsanityData to 14 Oct 2026

EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model

arXiv 2205.15278 ↗First posted 30 May 2022cs.CVcited by 151

Not on the map. Why?

Cited by 25 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although significant progress has been made to audio-driven talking face generation, existing methods either neglect facial emotion or cannot be applied to arbitrary subjects. In this paper, we propose the Emotion-Aware Motion Model (EAMM) to generate one-shot emotional talking faces by involving an emotion source video. Specifically, we first propose an Audio2Facial-Dynamics module, which renders talking faces from audio-driven unsupervised zero- and first-order key-points motion. Then through exploring the motion model's properties, we further propose an Implicit Emotion Displacement Learner to represent emotion-related facial dynamics as linearly additive displacements to the previously acquired motion representations. Comprehensive experiments demonstrate that by incorporating the results from both modules, our method can generate satisfactory talking face results on arbitrary subjects with realistic emotion patterns.

Key facts. Cited by 151 papers in the map (data through 14 Oct 2026). Builds on 42 of its 73 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

42 of 73 references.

References · 42

  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  2. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  3. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  4. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
  5. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  6. 2020First Order Motion Model for Image Animationcited by 704
  7. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
  8. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 632
  9. 2019Face X-ray for More General Face Forgery Detectioncited by 470
  10. 2017Look, Listen and Learncited by 458
  11. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 444
  12. 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 312
30 more
  1. 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
  2. 2020What makes fake images detectable? Understanding properties that generalizecited by 273
  3. 2018Deep Video Portraitscited by 267
  4. 2019Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Losscited by 252
  5. 2019Neural Voice Puppetry: Audio-driven Facial Reenactmentcited by 250
  6. 2018Cooperative Learning of Audio and Video Models from Self-Supervised Synchronizationcited by 249
  7. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  8. 2018Talking Face Generation by Adversarially Disentangled Audio-Visual Representationcited by 212
  9. 2018Animating Arbitrary Objects via Deep Motion Transfercited by 208
  10. 2021Audio-Driven Emotional Video Portraitscited by 180
  11. 2021Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motioncited by 149
  12. 2019Realistic Speech-Driven Facial Animation with GANscited by 142
  13. 2020Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatarscited by 118
  14. 2020Neural Head Reenactment with Latent Pose Descriptorscited by 106
  15. 2019Text-based Editing of Talking-head Videocited by 103
  16. 2017You said that?cited by 99
  17. 2018ReenactGAN: Learning to Reenact Faces via Boundary Transfercited by 85
  18. 2016Ambient Sound Provides Supervision for Visual Learningcited by 78
  19. 2018VisemeNet: Audio-Driven Animator-Centric Speech Animationcited by 63
  20. 2019Frame attention networks for facial expression recognition in videoscited by 56
  21. 2019Neural Style-Preserving Visual Dubbingcited by 47
  22. 2021Write-a-speaker: Text-based Emotional and Rhythmic Talking-head Generationcited by 47
  23. 2020Audio- and Gaze-driven Facial Animation of Codec Avatarscited by 46
  24. 2017Synthesizing Normalized Faces from Facial Identity Featurescited by 33
  25. 2020FACEGAN: Facial Attribute Controllable rEenactment GANcited by 32
  26. 2018End-to-End Speech-Driven Facial Animation with Temporal GANscited by 31
  27. 2020Mesh Guided One-shot Face Reenactment using Graph Convolutional Networkscited by 30
  28. 2019Animating Face using Disentangled Audio Representationscited by 24
  29. 2018Speech-Driven Expressive Talking Lips with Conditional Sequential Generative Adversarial Networkscited by 21
  30. 2020Iterative Text-based Editing of Talking-heads Using Neural Retargetingcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 9; 2023: 48; 2024: 45; 2025: 28; 2026: 21.

151 papers.

Sort:

By area:

  1. 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 382
  2. 2023CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Priorcited by 184
  3. 2023Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generationcited by 142
  4. 2023Identity-Preserving Talking Face Generation with Landmark and Appearance Priorscited by 98
  5. 2022Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesiscited by 93
  6. 2023StyleTalk: One-shot Talking Head Generation with Controllable Speaking Stylescited by 92
  7. 2023StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generatorcited by 89
  8. 2023SyncTalk: The Devil is in the Synchronization for Talking Head Synthesiscited by 86
  9. 2022Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decompositioncited by 81
  10. 2023Efficient Emotional Adaptation for Audio-Driven Talking-Head Generationcited by 81
  11. 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 80
  12. 2024EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesiscited by 80
38 more
  1. 2023FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusioncited by 78
  2. 2023DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Videocited by 74
  3. 2023SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Facescited by 68
  4. 2023Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generationcited by 61
  5. 2022Talking Head Generation with Probabilistic Audio-to-Visual Diffusion Priorscited by 55
  6. 2023FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Modelscited by 55
  7. 2024VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesiscited by 50
  8. 2022SPACE: Speech-driven Portrait Animation with Controllable Expressioncited by 49
  9. 2023GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generationcited by 42
  10. 2023The Power of Sound (TPoS): Audio Reactive Video Generation with Stable Diffusioncited by 41
  11. 2024FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portraitcited by 40
  12. 2024FlowVQTalker: High-Quality Emotional Talking Face Generation through Normalizing Flow and Quantizationcited by 39
  13. 2023High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learningcited by 37
  14. 2022Masked Lip-Sync Prediction by Audio-Visual Contextual Exploitation in Transformerscited by 33
  15. 2023VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactmentcited by 33
  16. 2024Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Stylecited by 33
  17. 2023TalkCLIP: Talking Head Generation with Text-Guided Expressive Speaking Stylescited by 32
  18. 2023FaceXHuBERT: Text-less Speech-driven E(X)pressive 3D Facial Animation Synthesis Using Self-Supervised Speech Representation Learningcited by 29
  19. 2023MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentionscited by 29
  20. 2023Speech Driven Video Editing via an Audio-Conditioned Diffusion Modelcited by 28
  21. 2023FT2TF: First-Person Statement Text-To-Talking Face Generationcited by 26
  22. 2024Say Anything with Any Stylecited by 26
  23. 2023DREAM-Talk: Diffusion-based Realistic Emotional Audio-driven Method for Single Image Talking Face Generationcited by 25
  24. 2024ScanTalk: 3D Talking Heads from Unregistered Scanscited by 24
  25. 2024FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audiocited by 20
  26. 2024DiffTED: One-shot Audio-driven TED Talk Video Generation with Diffusion-based Co-speech Gesturescited by 20
  27. 2024VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresencecited by 19
  28. 2024TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Modelcited by 19
  29. 2023Speech-Driven 3D Face Animation with Composite and Regional Facial Movementscited by 17
  30. 2025Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generationcited by 17
  31. 2023MFR-Net: Multi-faceted Responsive Listening Head Generation via Denoising Diffusion Modelcited by 16
  32. 2023DreamTalk: When Emotional Talking Head Generation Meets Diffusion Probabilistic Modelscited by 16
  33. 2023Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animationcited by 16
  34. 2024InstructAvatar: Text-Guided Emotion and Motion Control for Avatar Generationcited by 16
  35. 2024EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Headcited by 16
  36. 2024ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performercited by 16
  37. 2024RealTalk: Real-time and Realistic Audio-driven Face Generation with 3D Facial Prior-guided Identity Alignment Networkcited by 14
  38. 2024KMTalk: Speech-Driven 3D Facial Animation with Key Motion Embeddingcited by 14

50 most cited of 151 All 151 (JSON)