# MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targets

arXiv: https://arxiv.org/abs/1911.08139 · HTML: https://mlsanity.com/paper/1911.08139/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2019.

## Built on by (most cited first, 30 of 90)

- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [Thin-Plate Spline Motion Model for Image Animation](https://mlsanity.com/paper/2203.14367/) (2022)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion](https://mlsanity.com/paper/2107.09293/) (2021)
- [MegaPortraits: One-shot Megapixel Neural Head Avatars](https://mlsanity.com/paper/2207.07621/) (2022)
- [One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning](https://mlsanity.com/paper/2112.02749/) (2021)
- [Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forward](https://mlsanity.com/paper/2103.00484/) (2021)
- [MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation](https://mlsanity.com/paper/2212.08062/) (2022)
- [One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Field](https://mlsanity.com/paper/2304.05097/) (2023)
- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generation](https://mlsanity.com/paper/2307.09906/) (2023)
- [Magic-Me: Identity-Specific Video Customized Diffusion](https://mlsanity.com/paper/2402.09368/) (2024)
- [Countering Malicious DeepFakes: Survey, Battleground, and Horizon](https://mlsanity.com/paper/2103.00218/) (2021)
- [3D GAN Inversion for Controllable Portrait Image Animation](https://mlsanity.com/paper/2203.13441/) (2022)
- [VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis](https://mlsanity.com/paper/2403.08764/) (2024)
- [SPACE: Speech-driven Portrait Animation with Controllable Expression](https://mlsanity.com/paper/2211.09809/) (2022)
- [Face Animation with an Attribute-Guided Diffusion Model](https://mlsanity.com/paper/2304.03199/) (2023)
- [HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces](https://mlsanity.com/paper/2307.10797/) (2023)
- [Write-a-speaker: Text-based Emotional and Rhythmic Talking-head Generation](https://mlsanity.com/paper/2104.07995/) (2021)
- [DPE: Disentanglement of Pose and Expression for General Video Portrait Editing](https://mlsanity.com/paper/2301.06281/) (2023)
- [Implicit Warping for Animation with Image Sets](https://mlsanity.com/paper/2210.01794/) (2022)
- [Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields](https://mlsanity.com/paper/2206.08361/) (2022)
- [FSGANv2: Improved Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/2202.12972/) (2022)
- [Learned Spatial Representations for Few-shot Talking-Head Synthesis](https://mlsanity.com/paper/2104.14557/) (2021)
- [MegActor: Harness the Power of Raw Video for Vivid Portrait Animation](https://mlsanity.com/paper/2405.20851/) (2024)
- [Finding Directions in GAN's Latent Space for Neural Face Reenactment](https://mlsanity.com/paper/2202.00046/) (2022)
- [Towards Realistic Visual Dubbing with Heterogeneous Sources](https://mlsanity.com/paper/2201.06260/) (2022)
- [FaceController: Controllable Attribute Editing for Face in the Wild](https://mlsanity.com/paper/2102.11464/) (2021)
- [FNeVR: Neural Volume Rendering for Face Animation](https://mlsanity.com/paper/2209.10340/) (2022)
- [SAFA: Structure Aware Face Animation](https://mlsanity.com/paper/2111.04928/) (2021)

## Abstract (arXiv)

When there is a mismatch between the target identity and the driver identity, face reenactment suffers severe degradation in the quality of the result, especially in a few-shot setting. The identity preservation problem, where the model loses the detailed information of the target leading to a defective output, is the most common failure mode. The problem has several potential sources such as the identity of the driver leaking due to the identity mismatch, or dealing with unseen large poses. To overcome such problems, we introduce components that address the mentioned problem: image attention block, target feature alignment, and landmark transformer. Through attending and warping the relevant features, the proposed architecture, called MarioNETte, produces high-quality reenactments of unseen identities in a few-shot setting. In addition, the landmark transformer dramatically alleviates the identity preservation problem by isolating the expression geometry through landmark disentanglement. Comprehensive experiments are performed to verify that the proposed framework can generate highly realistic faces, outperforming all other baselines, even under a significant mismatch of facial characteristics between the target and the driver.