# EmoTalkingGaussian: Continuous Emotion-conditioned Talking Head Synthesis

arXiv: https://arxiv.org/abs/2502.00654 · HTML: https://mlsanity.com/paper/2502.00654/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Feb 2025.

## Built on by (most cited first, 4 of 4)

- [ESGaussianFace: Emotional and Stylized Audio-Driven Facial Animation via 3D Gaussian Splatting](https://mlsanity.com/paper/2601.01847/) (2026)
- [RSATalker: Realistic Socially-Aware Talking Head Generation for Multi-Turn Conversation](https://mlsanity.com/paper/2601.10606/) (2026)
- [Few-Shot Identity Adaptation for 3D Talking Heads via Global Gaussian Field](https://mlsanity.com/paper/2506.22044/) (2025)
- [FacEDiT: Unified Talking Face Editing and Generation via Facial Motion Infilling](https://mlsanity.com/paper/2512.14056/) (2025)

## Abstract (arXiv)

3D Gaussian splatting-based talking head synthesis has recently gained attention for its ability to render high-fidelity images with real-time inference speed. However, since it is typically trained on only a short video that lacks the diversity in facial emotions, the resultant talking heads struggle to represent a wide range of emotions. To address this issue, we propose a lip-aligned emotional face generator and leverage it to train our EmoTalkingGaussian model. It is able to manipulate facial emotions conditioned on continuous emotion values (i.e., valence and arousal); while retaining synchronization of lip movements with input audio. Additionally, to achieve the accurate lip synchronization for in-the-wild audio, we introduce a self-supervised learning method that leverages a text-to-speech network and a visual-audio synchronization network. We experiment our EmoTalkingGaussian on publicly available videos and have obtained better results than state-of-the-arts in terms of image quality (measured in PSNR, SSIM, LPIPS), emotion expression (measured in V-RMSE, A-RMSE, V-SA, A-SA, Emotion Accuracy), and lip synchronization (measured in LMD, Sync-E, Sync-C), respectively.