# Towards Open-Set Identity Preserving Face Synthesis

arXiv: https://arxiv.org/abs/1803.11182 · HTML: https://mlsanity.com/paper/1803.11182/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2018.

## Built on by (most cited first, 30 of 74)

- [SimSwap: An Efficient Framework For High Fidelity Face Swapping](https://mlsanity.com/paper/2106.06340/) (2021)
- [Exploring Temporal Coherence for More General Video Face Forgery Detection](https://mlsanity.com/paper/2108.06693/) (2021)
- [SynFace: Face Recognition with Synthetic Data](https://mlsanity.com/paper/2108.07960/) (2021)
- [One Shot Face Swapping on Megapixels](https://mlsanity.com/paper/2105.04932/) (2021)
- [Protecting Celebrities from DeepFake with Identity Consistency Transformer](https://mlsanity.com/paper/2203.01318/) (2022)
- [GAN-Control: Explicitly Controllable GANs](https://mlsanity.com/paper/2101.02477/) (2021)
- [Fine-Grained Face Swapping via Regional GAN Inversion](https://mlsanity.com/paper/2211.14068/) (2022)
- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [BlendFace: Re-designing Identity Encoders for Face-Swapping](https://mlsanity.com/paper/2307.10854/) (2023)
- [OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wild](https://mlsanity.com/paper/2107.14480/) (2021)
- [Region-Aware Face Swapping](https://mlsanity.com/paper/2203.04564/) (2022)
- [FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping](https://mlsanity.com/paper/2210.10473/) (2022)
- [STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection](https://mlsanity.com/paper/2306.02763/) (2023)
- [Initiative Defense against Facial Manipulation](https://mlsanity.com/paper/2112.10098/) (2021)
- [Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models](https://mlsanity.com/paper/2409.07269/) (2024)
- [High-Fidelity and Arbitrary Face Editing](https://mlsanity.com/paper/2103.15814/) (2021)
- [ADNet: Leveraging Error-Bias Towards Normal Direction in Face Alignment](https://mlsanity.com/paper/2109.05721/) (2021)
- [Smooth-Swap: A Simple Enhancement for Face-Swapping with Smoothness](https://mlsanity.com/paper/2112.05907/) (2021)
- [Finding Directions in GAN's Latent Space for Neural Face Reenactment](https://mlsanity.com/paper/2202.00046/) (2022)
- [On the use of automatically generated synthetic image datasets for benchmarking face recognition](https://mlsanity.com/paper/2106.04215/) (2021)
- [A Survey on Open Set Recognition](https://mlsanity.com/paper/2109.00893/) (2021)
- [Restricted Black-box Adversarial Attack Against DeepFake Face Swapping](https://mlsanity.com/paper/2204.12347/) (2022)
- [Deepfake Videos in the Wild: Analysis and Detection](https://mlsanity.com/paper/2103.04263/) (2021)
- [Boundary Guided Learning-Free Semantic Control with Diffusion Models](https://mlsanity.com/paper/2302.08357/) (2023)
- [Few-Shot Head Swapping in the Wild](https://mlsanity.com/paper/2204.13100/) (2022)
- [3D-FM GAN: Towards 3D-Controllable Face Manipulation](https://mlsanity.com/paper/2208.11257/) (2022)
- [Detecting Deepfakes Without Seeing Any](https://mlsanity.com/paper/2311.01458/) (2023)
- [Deepfake Forensics via An Adversarial Game](https://mlsanity.com/paper/2103.13567/) (2021)
- [Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval](https://mlsanity.com/paper/2107.06256/) (2021)
- [DeepBlur: A Simple and Effective Method for Natural Image Obfuscation](https://mlsanity.com/paper/2104.02655/) (2021)

## Abstract (arXiv)

We propose a framework based on Generative Adversarial Networks to disentangle the identity and attributes of faces, such that we can conveniently recombine different identities and attributes for identity preserving face synthesis in open domains. Previous identity preserving face synthesis processes are largely confined to synthesizing faces with known identities that are already in the training dataset. To synthesize a face with identity outside the training dataset, our framework requires one input image of that subject to produce an identity vector, and any other input face image to extract an attribute vector capturing, e.g., pose, emotion, illumination, and even the background. We then recombine the identity vector and the attribute vector to synthesize a new face of the subject with the extracted attribute. Our proposed framework does not need to annotate the attributes of faces in any way. It is trained with an asymmetric loss function to better preserve the identity and stabilize the training process. It can also effectively leverage large amounts of unlabeled training face images to further improve the fidelity of the synthesized faces for subjects that are not presented in the labeled training face dataset. Our experiments demonstrate the efficacy of the proposed framework. We also present its usage in a much broader set of applications including face frontalization, face attribute morphing, and face adversarial example detection.