# Deep Learning Based Facial Retargeting Using Local Patches

arXiv: https://arxiv.org/abs/2601.08429 · HTML: https://mlsanity.com/paper/2601.08429/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 57 references that are in the map.
- First posted on arXiv 13 Jan 2026.

## Builds on (most cited first, 14 of 14)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control](https://mlsanity.com/paper/2407.03168/) (2024)
- [DeepFaceLab: Integrated, flexible and extensible face-swapping framework](https://mlsanity.com/paper/2005.05535/) (2020)
- [Towards High-fidelity Nonlinear 3D Face Morphable Model](https://mlsanity.com/paper/1904.04933/) (2019)
- [Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis](https://mlsanity.com/paper/1704.04086/) (2017)
- [GAN-Supervised Dense Visual Alignment](https://mlsanity.com/paper/2112.05143/) (2021)
- [Neural Congealing: Aligning Images to a Joint Semantic Atlas](https://mlsanity.com/paper/2302.03956/) (2023)
- [Joint Face Detection and Facial Motion Retargeting for Multiple Faces](https://mlsanity.com/paper/1902.10744/) (2019)
- [Facial Expression Retargeting from Human to Avatar Made Easy](https://mlsanity.com/paper/2008.05110/) (2020)
- [Learning Stylized Character Expressions from Humans](https://mlsanity.com/paper/1911.08591/) (2019)

## Built on by (most cited first, 1 of 1)

- [AnyTalk: Speech Animation for Arbitrary Characters Leveraging a Video Generation Model](https://mlsanity.com/paper/2608.16143/) (2026)

## Abstract (arXiv)

In the era of digital animation, the quest to produce lifelike facial animations for virtual characters has led to the development of various retargeting methods. While the retargeting facial motion between models of similar shapes has been very successful, challenges arise when the retargeting is performed on stylized or exaggerated 3D characters that deviate significantly from human facial structures. In this scenario, it is important to consider the target character's facial structure and possible range of motion to preserve the semantics assumed by the original facial motions after the retargeting. To achieve this, we propose a local patch-based retargeting method that transfers facial animations captured in a source performance video to a target stylized 3D character. Our method consists of three modules. The Automatic Patch Extraction Module extracts local patches from the source video frame. These patches are processed through the Reenactment Module to generate correspondingly re-enacted target local patches. The Weight Estimation Module calculates the animation parameters for the target character at every frame for the creation of a complete facial animation sequence. Extensive experiments demonstrate that our method can successfully transfer the semantic meaning of source facial expressions to stylized characters with considerable variations in facial feature proportion.