# High Fidelity Face Manipulation with Extreme Poses and Expressions

arXiv: https://arxiv.org/abs/1903.12003 · HTML: https://mlsanity.com/paper/1903.12003/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2019.

## Built on by (most cited first, 8 of 8)

- [Few-shot Backdoor Defense Using Shapley Estimation](https://mlsanity.com/paper/2112.14889/) (2021)
- [Dual Spoof Disentanglement Generation for Face Anti-spoofing with Depth Uncertainty Learning](https://mlsanity.com/paper/2112.00568/) (2021)
- [POCE: Pose-Controllable Expression Editing](https://mlsanity.com/paper/2304.08938/) (2023)
- [A comprehensive survey on semantic facial attribute editing using generative adversarial networks](https://mlsanity.com/paper/2205.10587/) (2022)
- [Deepfake Detection via Joint Unsupervised Reconstruction and Supervised Classification](https://mlsanity.com/paper/2211.13424/) (2022)
- [PriorityCut: Occlusion-guided Regularization for Warp-based Image Animation](https://mlsanity.com/paper/2103.11600/) (2021)
- [Escaping Data Scarcity for High-Resolution Heterogeneous Face Hallucination](https://mlsanity.com/paper/2203.16669/) (2022)
- [IFaceUV: Intuitive Motion Facial Image Generation by Identity Preservation via UV map](https://mlsanity.com/paper/2306.04957/) (2023)

## Abstract (arXiv)

Face manipulation has shown remarkable advances with the flourish of Generative Adversarial Networks. However, due to the difficulties of controlling structures and textures, it is challenging to model poses and expressions simultaneously, especially for the extreme manipulation at high-resolution. In this paper, we propose a novel framework that simplifies face manipulation into two correlated stages: a boundary prediction stage and a disentangled face synthesis stage. The first stage models poses and expressions jointly via boundary images. Specifically, a conditional encoder-decoder network is employed to predict the boundary image of the target face in a semi-supervised way. Pose and expression estimators are introduced to improve the prediction performance. In the second stage, the predicted boundary image and the input face image are encoded into the structure and the texture latent space by two encoder networks, respectively. A proxy network and a feature threshold loss are further imposed to disentangle the latent space. Furthermore, due to the lack of high-resolution face manipulation databases to verify the effectiveness of our method, we collect a new high-quality Multi-View Face (MVF-HQ) database. It contains 120,283 images at 6000x4000 resolution from 479 identities with diverse poses, expressions, and illuminations. MVF-HQ is much larger in scale and much higher in resolution than publicly available high-resolution face manipulation databases. We will release MVF-HQ soon to push forward the advance of face manipulation. Qualitative and quantitative experiments on four databases show that our method dramatically improves the synthesis quality.