EditEmoTalk: Controllable Speech-Driven 3D Facial Animation with Continuous Expression Editing
Not on the map. Why?
Abstract (arXiv)
Speech-driven 3D facial animation aims to generate realistic and expressive facial motions directly from audio. While recent methods achieve high-quality lip synchronization, they often rely on discrete emotion categories, limiting continuous and fine-grained emotional control. We present EditEmoTalk, a controllable speech-driven 3D facial animation framework with continuous emotion editing. The key idea is a boundary-aware semantic embedding that learns the normal directions of inter-emotion decision boundaries, enabling a continuous expression manifold for smooth emotion manipulation. Moreover, we introduce an emotional consistency loss that enforces semantic alignment between the generated motion dynamics and the target emotion embedding through a mapping network, ensuring faithful emotional expression. Extensive experiments demonstrate that EditEmoTalk achieves superior controllability, expressiveness, and generalization while maintaining accurate lip synchronization. Code and pretrained models will be released.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 15 Jan 2026.
Builds on
24 of 41 references.
References · 24
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
- 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 257
- 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
- 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 208
- 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 196
- 2021MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglementcited by 189
- 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
- 2023CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Priorcited by 183
- 2023EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animationcited by 141
- 2023Collaborative Diffusion for Multi-Modal Face Generation and Editingcited by 116
12 more
- 2023Emotional Speech-Driven Animation with Content-Emotion Disentanglementcited by 98
- 2023DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Modelscited by 81
- 2024SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesiscited by 81
- 2023FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusioncited by 77
- 2022Imitator: Personalized Speech-driven 3D Facial Animationcited by 73
- 2019Disentangled Representation Learning for 3D Face Shapecited by 48
- 2022Face Generation and Editing with StyleGAN: A Surveycited by 27
- 2024RealTalk: Real-time and Realistic Audio-driven Face Generation with 3D Facial Prior-guided Identity Alignment Networkcited by 14
- 2024DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animationcited by 12
- 2025PESTalk: Speech-Driven 3D Facial Animation with Personalized Emotional Stylescited by 4
- 2024JambaTalk: Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Modelcited by 3
- 2024CSTalk: Correlation Supervised Speech-driven 3D Emotional Facial Animation Generationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.