EmoZone-Talker: Regional Semantic Control of Audio-Driven 3DGS Talking Heads via Facial Action Units
Not on the map. Why?
Abstract (arXiv)
3D Gaussian Splatting (3DGS) has shown strong potential for high-fidelity talking head synthesis. However, enabling fine-grained, interpretable, and editable facial expression control remains fundamentally challenging due to intrinsic conflicts between speech-driven facial dynamics and explicit expression signals. Existing methods rely on implicit multimodal fusion, leading to spatial entanglement and temporal instability. We present EmoZone-Talker, a novel framework that reformulates audio-driven facial animation as a structured spatial-temporal coordination problem under cross-modal conflicts. Our approach introduces an explicit spatial disentanglement and temporal dynamics modeling of facial motion. Specifically, we propose Synergy Zones with Prioritized Attention Bias (SZ-PAB) to explicitly decouple modality contributions via region-wise constraints guided by anatomical priors, and a Channel-Independent Temporal AU Encoder (CIT-AE) to model temporally coherent AU dynamics. By integrating these representations into 3D Gaussian deformation, EmoZone-Talker enables precise and interpretable control over facial expressions. Extensive experiments demonstrate that our method improves expression controllability and realism, with notable gains in upper-face accuracy and temporal coherence, while preserving high rendering quality and accurate lip synchronization. Code will be publicly released to facilitate reproducibility and further research.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 49 references that are in the map. First posted on arXiv 14 Jun 2026.
Builds on
29 of 49 references.
References · 29
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
- 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 492
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 382
- 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
- 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 312
- 2019Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Losscited by 252
- 2021Audio-Driven Emotional Video Portraitscited by 180
- 2022EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Modelcited by 151
- 2023GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesiscited by 131
17 more
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 122
- 2023StyleTalk: One-shot Talking Head Generation with Controllable Speaking Stylescited by 92
- 2022Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decompositioncited by 81
- 2023Efficient Emotional Adaptation for Audio-Driven Talking-Head Generationcited by 81
- 2023Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesiscited by 72
- 2024TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splattingcited by 60
- 2018Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignmentcited by 55
- 2024Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidancecited by 39
- 2024EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Headcited by 16
- 2024Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesiscited by 12
- 2025Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Controlcited by 12
- 2025Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generationcited by 11
- 2025Monocular and Generalizable Gaussian Talking Head Animationcited by 10
- 2025Exploring Timeline Control for Facial Motion Generationcited by 4
- 2025PESTalk: Speech-Driven 3D Facial Animation with Personalized Emotional Stylescited by 4
- 2025PTalker: Personalized Speech-Driven 3D Talking Head Animation via Style Disentanglement and Modality Alignmentcited by 1
- 2026Toward Fine-Grained Facial Control in 3D Talking Head Generationcited by 1
Built on by
Not cited yet.