# Learning Semantic Facial Descriptors for Accurate Face Animation

arXiv: https://arxiv.org/abs/2501.17718 · HTML: https://mlsanity.com/paper/2501.17718/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 29 Jan 2025.

## Builds on (most cited first, 21 of 21)

- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering](https://mlsanity.com/paper/2109.08379/) (2021)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [Latent Image Animator: Learning to Animate Images via Latent Space Navigation](https://mlsanity.com/paper/2203.09043/) (2022)
- [StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN](https://mlsanity.com/paper/2203.04036/) (2022)
- [HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping](https://mlsanity.com/paper/2106.09965/) (2021)
- [ReenactGAN: Learning to Reenact Faces via Boundary Transfer](https://mlsanity.com/paper/1807.11079/) (2018)
- [3D face reconstruction with dense landmarks](https://mlsanity.com/paper/2204.02776/) (2022)
- [MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation](https://mlsanity.com/paper/2212.08062/) (2022)
- [Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generation](https://mlsanity.com/paper/2307.09906/) (2023)
- [NOFA: NeRF-based One-shot Facial Avatar Reconstruction](https://mlsanity.com/paper/2307.03441/) (2023)
- [DiffusionRet: Generative Text-Video Retrieval with Diffusion Model](https://mlsanity.com/paper/2303.09867/) (2023)
- [DPE: Disentanglement of Pose and Expression for General Video Portrait Editing](https://mlsanity.com/paper/2301.06281/) (2023)
- [Mesh Guided One-shot Face Reenactment using Graph Convolutional Networks](https://mlsanity.com/paper/2008.07783/) (2020)
- [Toward 3D Spatial Reasoning for Human-like Text-based Visual Question Answering](https://mlsanity.com/paper/2209.10326/) (2022)
- [Audio-Visual Face Reenactment](https://mlsanity.com/paper/2210.02755/) (2022)
- [LI-Net: Large-Pose Identity-Preserving Face Reenactment Network](https://mlsanity.com/paper/2104.02850/) (2021)

## Built on by (most cited first, 2 of 2)

- [Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesis](https://mlsanity.com/paper/2601.17687/) (2026)
- [Tikhonov-Stabilized Bezier Representation Forecasting for Training-free Diffusion Acceleration](https://mlsanity.com/paper/2610.16448/) (2026)

## Abstract (arXiv)

Face animation is a challenging task. Existing model-based methods (utilizing 3DMMs or landmarks) often result in a model-like reconstruction effect, which doesn't effectively preserve identity. Conversely, model-free approaches face challenges in attaining a decoupled and semantically rich feature space, thereby making accurate motion transfer difficult to achieve. We introduce the semantic facial descriptors in learnable disentangled vector space to address the dilemma. The approach involves decoupling the facial space into identity and motion subspaces while endowing each of them with semantics by learning complete orthogonal basis vectors. We obtain basis vector coefficients by employing an encoder on the source and driving faces, leading to effective facial descriptors in the identity and motion subspaces. Ultimately, these descriptors can be recombined as latent codes to animate faces. Our approach successfully addresses the issue of model-based methods' limitations in high-fidelity identity and the challenges faced by model-free methods in accurate motion transfer. Extensive experiments are conducted on three challenging benchmarks (i.e. VoxCeleb, HDTF, CelebV). Comprehensive quantitative and qualitative results demonstrate that our model outperforms SOTA methods with superior identity preservation and motion transfer.