# EmoZone-Talker: Regional Semantic Control of Audio-Driven 3DGS Talking Heads via Facial Action Units

arXiv: https://arxiv.org/abs/2606.15848 · HTML: https://mlsanity.com/paper/2606.15848/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 14 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation](https://mlsanity.com/paper/2211.12194/) (2022)
- [AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis](https://mlsanity.com/paper/2103.11078/) (2021)
- [MakeItTalk: Speaker-Aware Talking-Head Animation](https://mlsanity.com/paper/2004.12992/) (2020)
- [Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss](https://mlsanity.com/paper/1905.03820/) (2019)
- [Audio-Driven Emotional Video Portraits](https://mlsanity.com/paper/2104.07452/) (2021)
- [EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model](https://mlsanity.com/paper/2205.15278/) (2022)
- [GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis](https://mlsanity.com/paper/2301.13430/) (2023)
- [Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer](https://mlsanity.com/paper/2412.00733/) (2024)
- [StyleTalk: One-shot Talking Head Generation with Controllable Speaking Styles](https://mlsanity.com/paper/2301.01081/) (2023)
- [Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition](https://mlsanity.com/paper/2211.12368/) (2022)
- [Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation](https://mlsanity.com/paper/2309.04946/) (2023)
- [Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis](https://mlsanity.com/paper/2307.09323/) (2023)
- [TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splatting](https://mlsanity.com/paper/2404.15264/) (2024)
- [Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment](https://mlsanity.com/paper/1803.05588/) (2018)
- [Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance](https://mlsanity.com/paper/2401.15687/) (2024)
- [EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Head](https://mlsanity.com/paper/2408.00297/) (2024)
- [Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis](https://mlsanity.com/paper/2402.17364/) (2024)
- [Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Control](https://mlsanity.com/paper/2503.14517/) (2025)
- [Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation](https://mlsanity.com/paper/2504.18087/) (2025)
- [Monocular and Generalizable Gaussian Talking Head Animation](https://mlsanity.com/paper/2504.00665/) (2025)
- [Exploring Timeline Control for Facial Motion Generation](https://mlsanity.com/paper/2505.20861/) (2025)
- [PESTalk: Speech-Driven 3D Facial Animation with Personalized Emotional Styles](https://mlsanity.com/paper/2512.05121/) (2025)
- [PTalker: Personalized Speech-Driven 3D Talking Head Animation via Style Disentanglement and Modality Alignment](https://mlsanity.com/paper/2512.22602/) (2025)
- [Toward Fine-Grained Facial Control in 3D Talking Head Generation](https://mlsanity.com/paper/2602.09736/) (2026)

## Abstract (arXiv)

3D Gaussian Splatting (3DGS) has shown strong potential for high-fidelity talking head synthesis. However, enabling fine-grained, interpretable, and editable facial expression control remains fundamentally challenging due to intrinsic conflicts between speech-driven facial dynamics and explicit expression signals. Existing methods rely on implicit multimodal fusion, leading to spatial entanglement and temporal instability. We present EmoZone-Talker, a novel framework that reformulates audio-driven facial animation as a structured spatial-temporal coordination problem under cross-modal conflicts. Our approach introduces an explicit spatial disentanglement and temporal dynamics modeling of facial motion. Specifically, we propose Synergy Zones with Prioritized Attention Bias (SZ-PAB) to explicitly decouple modality contributions via region-wise constraints guided by anatomical priors, and a Channel-Independent Temporal AU Encoder (CIT-AE) to model temporally coherent AU dynamics. By integrating these representations into 3D Gaussian deformation, EmoZone-Talker enables precise and interpretable control over facial expressions. Extensive experiments demonstrate that our method improves expression controllability and realism, with notable gains in upper-face accuracy and temporal coherence, while preserving high rendering quality and accurate lip synchronization. Code will be publicly released to facilitate reproducibility and further research.