# PESTalk: Speech-Driven 3D Facial Animation with Personalized Emotional Styles

arXiv: https://arxiv.org/abs/2512.05121 · HTML: https://mlsanity.com/paper/2512.05121/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement](https://mlsanity.com/paper/2104.08223/) (2021)
- [CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior](https://mlsanity.com/paper/2301.02379/) (2023)
- [EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation](https://mlsanity.com/paper/2303.11089/) (2023)
- [Emotional Speech-Driven Animation with Content-Emotion Disentanglement](https://mlsanity.com/paper/2306.08990/) (2023)
- [DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models](https://mlsanity.com/paper/2310.00434/) (2023)
- [FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion](https://mlsanity.com/paper/2309.11306/) (2023)
- [SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces](https://mlsanity.com/paper/2306.10799/) (2023)
- [Deep Learning for Micro-expression Recognition: A Survey](https://mlsanity.com/paper/2107.02823/) (2021)
- [MEMO: Memory-Guided Diffusion for Expressive Talking Video Generation](https://mlsanity.com/paper/2412.04448/) (2024)
- [DiffusionTalker: Personalization and Acceleration for Speech-Driven 3D Face Diffuser](https://mlsanity.com/paper/2311.16565/) (2023)
- [Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2312.10877/) (2023)
- [3D-TalkEmo: Learning to Synthesize 3D Emotional Talking Head](https://mlsanity.com/paper/2104.12051/) (2021)
- [A Comparison of Metric Learning Loss Functions for End-To-End Speaker Verification](https://mlsanity.com/paper/2003.14021/) (2020)
- [SAiD: Speech-driven Blendshape Facial Animation with Diffusion](https://mlsanity.com/paper/2401.08655/) (2023)
- [GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer](https://mlsanity.com/paper/2408.01826/) (2024)
- [PMMTalk: Speech-Driven 3D Facial Animation from Complementary Pseudo Multi-modal Features](https://mlsanity.com/paper/2312.02781/) (2023)
- [PortraitTalk: Towards Customizable One-Shot Audio-to-Talking Face Generation](https://mlsanity.com/paper/2412.07754/) (2024)

## Built on by (most cited first, 4 of 4)

- [EditEmoTalk: Controllable Speech-Driven 3D Facial Animation with Continuous Expression Editing](https://mlsanity.com/paper/2601.10000/) (2026)
- [EmoZone-Talker: Regional Semantic Control of Audio-Driven 3DGS Talking Heads via Facial Action Units](https://mlsanity.com/paper/2606.15848/) (2026)
- [Human4K: A Large-Scale 4K Multi-View Mocap Dataset for Whole-Body 3D Human Reconstruction](https://mlsanity.com/paper/2607.13646/) (2026)
- [PointCHR: Point Cloud Analysis via Curvature-Aware Hyperbolic Rectification](https://mlsanity.com/paper/2607.24052/) (2026)

## Abstract (arXiv)

PESTalk is a novel method for generating 3D facial animations with personalized emotional styles directly from speech. It overcomes key limitations of existing approaches by introducing a Dual-Stream Emotion Extractor (DSEE) that captures both time and frequency-domain audio features for fine-grained emotion analysis, and an Emotional Style Modeling Module (ESMM) that models individual expression patterns based on voiceprint characteristics. To address data scarcity, the method leverages a newly constructed 3D-EmoStyle dataset. Evaluations demonstrate that PESTalk outperforms state-of-the-art methods in producing realistic and personalized facial animations.