# HeadOn: Real-time Reenactment of Human Portrait Videos

arXiv: https://arxiv.org/abs/1805.11729 · HTML: https://mlsanity.com/paper/1805.11729/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2018.

## Built on by (most cited first, 11 of 11)

- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forward](https://mlsanity.com/paper/2103.00484/) (2021)
- [MEGANE: Morphable Eyeglass and Avatar Network](https://mlsanity.com/paper/2302.04868/) (2023)
- [Egocentric Videoconferencing](https://mlsanity.com/paper/2107.03109/) (2021)
- [Metric Learning for Anti-Compression Facial Forgery Detection](https://mlsanity.com/paper/2103.08397/) (2021)
- [MaskRenderer: 3D-Infused Multi-Mask Realistic Face Reenactment](https://mlsanity.com/paper/2309.05095/) (2023)
- [2CET-GAN: Pixel-Level GAN Model for Human Facial Expression Transfer](https://mlsanity.com/paper/2211.11570/) (2022)
- [Dynamic Neural Portraits](https://mlsanity.com/paper/2211.13994/) (2022)
- [High-fidelity Interpretable Inverse Rig: An Accurate and Sparse Solution Optimizing the Quartic Blendshape Model](https://mlsanity.com/paper/2302.04820/) (2023)
- [Real-centric Consistency Learning for Deepfake Detection](https://mlsanity.com/paper/2205.07201/) (2022)

## Abstract (arXiv)

We propose HeadOn, the first real-time source-to-target reenactment approach for complete human portrait videos that enables transfer of torso and head motion, face expression, and eye gaze. Given a short RGB-D video of the target actor, we automatically construct a personalized geometry proxy that embeds a parametric head, eye, and kinematic torso model. A novel real-time reenactment algorithm employs this proxy to photo-realistically map the captured motion from the source actor to the target actor. On top of the coarse geometric proxy, we propose a video-based rendering technique that composites the modified target portrait video via view- and pose-dependent texturing, and creates photo-realistic imagery of the target actor under novel torso and head poses, facial expressions, and gaze directions. To this end, we propose a robust tracking of the face and torso of the source actor. We extensively evaluate our approach and show significant improvements in enabling much greater flexibility in creating realistic reenacted output videos.