mlsanityData to 14 Oct 2026

FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing

arXiv 2505.22141 ↗First posted 28 May 2025cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Recent advances in audio-driven talking head generation have achieved impressive results in lip synchronization and emotional expression. However, they largely overlook the crucial task of facial attribute editing. This capability is indispensable for achieving deep personalization and expanding the range of practical applications, including user-tailored digital avatars, engaging online education content, and brand-specific digital customer service. In these key domains, flexible adjustment of visual attributes, such as hairstyle, accessories, and subtle facial features, is essential for aligning with user preferences, reflecting diverse brand identities and adapting to varying contextual demands. In this paper, we present FaceEditTalker, a unified framework that enables controllable facial attribute manipulation while generating high-quality, audio-synchronized talking head videos. Our method consists of two key components: an image feature space editing module, which extracts semantic and detail features and allows flexible control over attributes like expression, hairstyle, and accessories; and an audio-driven video generation module, which fuses these edited features with audio-guided facial landmarks to drive a diffusion-based generator. This design ensures temporal coherence, visual fidelity, and identity preservation across frames. Extensive experiments on public datasets demonstrate that our method achieves comparable or superior performance to representative baseline methods in lip-sync accuracy, video quality, and attribute controllability. Project page: https://peterfanfan.github.io/FaceEditTalker/

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 55 of its 71 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

55 of 71 references.

References · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Implicit Modelscited by 9,781
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  6. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  7. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  8. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  9. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
  10. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  11. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 449
  12. 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 382
38 more
  1. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  2. 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 312
  3. 2019Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Losscited by 252
  4. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 235
  5. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 202
  6. 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
  7. 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 194
  8. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 176
  9. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  10. 2018Lip Movements Generation at a Glancecited by 155
  11. 2023DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animationcited by 136
  12. 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 122
  13. 2022VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wildcited by 106
  14. 2024Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animationcited by 97
  15. 2023StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generatorcited by 89
  16. 2023DiffusionRig: Learning Personalized Priors for Facial Appearance Editingcited by 75
  17. 2023Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesiscited by 72
  18. 2024EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animationcited by 72
  19. 2022Stitch it in Time: GAN-Based Facial Editing of Real Videoscited by 68
  20. 2021A Latent Transformer for Disentangled Face Editing in Images and Videoscited by 67
  21. 2022SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memorycited by 65
  22. 2024TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splattingcited by 60
  23. 2025HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animationcited by 59
  24. 2025Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Modelcited by 49
  25. 2022Diffusion Video Autoencoders: Toward Temporally Consistent Face Video Editing via Disentangled Video Encodingcited by 39
  26. 2025SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformerscited by 33
  27. 2022Temporally Consistent Semantic Video Editingcited by 28
  28. 2025InsTaG: Learning Personalized 3D Talking Head from Few-Second Videocited by 18
  29. 2025Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generationcited by 17
  30. 2023AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesiscited by 16
  31. 2024MimicTalk: Mimicking a personalized and expressive 3D talking face in minutescited by 16
  32. 2024EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusioncited by 16
  33. 2024DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactmentcited by 14
  34. 2024SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesiscited by 14
  35. 2023Audio-driven Talking Face Generation with Stabilized Synchronization Losscited by 13
  36. 2023Identity-Preserving Aging of Face Images via Latent Diffusion Modelscited by 11
  37. 2025Monocular and Generalizable Gaussian Talking Head Animationcited by 10
  38. 2025AV-Flow: Transforming Text to Audio-Visual Human-like Interactionscited by 9

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.