# Neural Head Reenactment with Latent Pose Descriptors

arXiv: https://arxiv.org/abs/2004.12000 · HTML: https://mlsanity.com/paper/2004.12000/

## Key facts

- Cited by 106 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2020.

## Built on by (most cited first, 30 of 106)

- [Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation](https://mlsanity.com/paper/2104.11116/) (2021)
- [PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering](https://mlsanity.com/paper/2109.08379/) (2021)
- [Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation](https://mlsanity.com/paper/2406.08801/) (2024)
- [VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time](https://mlsanity.com/paper/2404.10667/) (2024)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN](https://mlsanity.com/paper/2203.04036/) (2022)
- [EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model](https://mlsanity.com/paper/2205.15278/) (2022)
- [Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion](https://mlsanity.com/paper/2107.09293/) (2021)
- [MegaPortraits: One-shot Megapixel Neural Head Avatars](https://mlsanity.com/paper/2207.07621/) (2022)
- [Generalizable and Animatable Gaussian Head Avatar](https://mlsanity.com/paper/2410.07971/) (2024)
- [Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesis](https://mlsanity.com/paper/2211.14506/) (2022)
- [StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator](https://mlsanity.com/paper/2305.05445/) (2023)
- [One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Field](https://mlsanity.com/paper/2304.05097/) (2023)
- [EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatars](https://mlsanity.com/paper/2404.19110/) (2024)
- [Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizer](https://mlsanity.com/paper/2403.13570/) (2024)
- [Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templates](https://mlsanity.com/paper/2108.08020/) (2021)
- [Talking Head Generation with Probabilistic Audio-to-Visual Diffusion Priors](https://mlsanity.com/paper/2212.04248/) (2022)
- [Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data](https://mlsanity.com/paper/2311.18729/) (2023)
- [CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models](https://mlsanity.com/paper/2412.12093/) (2024)
- [SPACE: Speech-driven Portrait Animation with Controllable Expression](https://mlsanity.com/paper/2211.09809/) (2022)
- [Face Animation with an Attribute-Guided Diffusion Model](https://mlsanity.com/paper/2304.03199/) (2023)
- [HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces](https://mlsanity.com/paper/2307.10797/) (2023)
- [DPE: Disentanglement of Pose and Expression for General Video Portrait Editing](https://mlsanity.com/paper/2301.06281/) (2023)
- [Implicit Warping for Animation with Image Sets](https://mlsanity.com/paper/2210.01794/) (2022)
- [RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars](https://mlsanity.com/paper/2305.13353/) (2023)
- [GAIA: Zero-shot Talking Avatar Generation](https://mlsanity.com/paper/2311.15230/) (2023)
- [LAM: Large Avatar Model for One-shot Animatable Gaussian Head](https://mlsanity.com/paper/2502.17796/) (2025)
- [High-Fidelity and Freely Controllable Talking Head Video Generation](https://mlsanity.com/paper/2304.10168/) (2023)
- [StyleSwap: Style-Based Generator Empowers Robust Face Swapping](https://mlsanity.com/paper/2209.13514/) (2022)
- [Structure-Aware Motion Transfer with Deformable Anchor Model](https://mlsanity.com/paper/2204.05018/) (2022)

## Abstract (arXiv)

We propose a neural head reenactment system, which is driven by a latent pose representation and is capable of predicting the foreground segmentation alongside the RGB image. The latent pose representation is learned as a part of the entire reenactment system, and the learning process is based solely on image reconstruction losses. We show that despite its simplicity, with a large and diverse enough training dataset, such learning successfully decomposes pose from identity. The resulting system can then reproduce mimics of the driving person and, furthermore, can perform cross-person reenactment. Additionally, we show that the learned descriptors are useful for other pose-related tasks, such as keypoint prediction and pose-based retrieval.