# PTalker: Personalized Speech-Driven 3D Talking Head Animation via Style Disentanglement and Modality Alignment

arXiv: https://arxiv.org/abs/2512.22602 · HTML: https://mlsanity.com/paper/2512.22602/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 27 Dec 2025.

## Builds on (most cited first, 20 of 20)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement](https://mlsanity.com/paper/2104.08223/) (2021)
- [CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior](https://mlsanity.com/paper/2301.02379/) (2023)
- [EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation](https://mlsanity.com/paper/2303.11089/) (2023)
- [DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models](https://mlsanity.com/paper/2310.00434/) (2023)
- [FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion](https://mlsanity.com/paper/2309.11306/) (2023)
- [Imitator: Personalized Speech-driven 3D Facial Animation](https://mlsanity.com/paper/2301.00023/) (2022)
- [VisemeNet: Audio-Driven Animator-Centric Speech Animation](https://mlsanity.com/paper/1805.09488/) (2018)
- [Audio- and Gaze-driven Facial Animation of Codec Avatars](https://mlsanity.com/paper/2008.05023/) (2020)
- [Speech-Driven 3D Face Animation with Composite and Regional Facial Movements](https://mlsanity.com/paper/2308.05428/) (2023)
- [Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2312.10877/) (2023)
- [MMHead: Towards Fine-grained Multi-modal 3D Facial Animation](https://mlsanity.com/paper/2410.07757/) (2024)
- [Learning Audio-Driven Viseme Dynamics for 3D Face Animation](https://mlsanity.com/paper/2301.06059/) (2023)
- [Meta-Learning Empowered Meta-Face: Personalized Speaking Style Adaptation for Audio-Driven 3D Talking Face Animation](https://mlsanity.com/paper/2408.09357/) (2024)
- [3DiFACE: Synthesizing and Editing Holistic 3D Facial Animation](https://mlsanity.com/paper/2509.26233/) (2025)
- [Speaker Independent and Multilingual/Mixlingual Speech-Driven Talking Head Generation Using Phonetic Posteriorgrams](https://mlsanity.com/paper/2006.11610/) (2020)
- [DualTalker: A Cross-Modal Dual Learning Approach for Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2311.04766/) (2023)
- [Content and Style Aware Audio-Driven Facial Animation](https://mlsanity.com/paper/2408.07005/) (2024)
- [StyleSpeaker: Audio-Enhanced Fine-Grained Style Modeling for Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2503.09852/) (2025)

## Built on by (most cited first, 1 of 1)

- [EmoZone-Talker: Regional Semantic Control of Audio-Driven 3DGS Talking Heads via Facial Action Units](https://mlsanity.com/paper/2606.15848/) (2026)

## Abstract (arXiv)

Speech-driven 3D talking head generation aims to produce lifelike facial animations precisely synchronized with speech. While considerable progress has been made in achieving high lip-synchronization accuracy, existing methods largely overlook the intricate nuances of individual speaking styles, which limits personalization and realism. In this work, we present a novel framework for personalized 3D talking head animation, namely "PTalker". This framework preserves speaking style through style disentanglement from audio and facial motion sequences and enhances lip-synchronization accuracy through a three-level alignment mechanism between audio and mesh modalities. Specifically, to effectively disentangle style and content, we design disentanglement constraints that encode driven audio and motion sequences into distinct style and content spaces to enhance speaking style representation. To improve lip-synchronization accuracy, we adopt a modality alignment mechanism incorporating three aspects: spatial alignment using Graph Attention Networks to capture vertex connectivity in the 3D mesh structure, temporal alignment using cross-attention to capture and synchronize temporal dependencies, and feature alignment by top-k bidirectional contrastive losses and KL divergence constraints to ensure consistency between speech and mesh modalities. Extensive qualitative and quantitative experiments on public datasets demonstrate that PTalker effectively generates realistic, stylized 3D talking heads that accurately match identity-specific speaking styles, outperforming state-of-the-art methods. The source code and supplementary videos are available at: PTalker.