# GMFIM: A Generative Mask-guided Facial Image Manipulation Model for Privacy Preservation

arXiv: https://arxiv.org/abs/2201.03353 · HTML: https://mlsanity.com/paper/2201.03353/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 53 references that are in the map.
- First posted on arXiv 10 Jan 2022.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Neural Photo Editing with Introspective Adversarial Networks](https://mlsanity.com/paper/1609.07093/) (2016)
- [DeepPrivacy: A Generative Adversarial Network for Face Anonymization](https://mlsanity.com/paper/1909.04538/) (2019)
- [A Survey on Generative Adversarial Networks: Variants, Applications, and Training](https://mlsanity.com/paper/2006.05132/) (2020)
- [A Study of Face Obfuscation in ImageNet](https://mlsanity.com/paper/2103.06191/) (2021)
- [Patch-Based Image Inpainting with Generative Adversarial Networks](https://mlsanity.com/paper/1803.07422/) (2018)
- [Natural and Effective Obfuscation by Head Inpainting](https://mlsanity.com/paper/1711.09001/) (2017)
- [AdvFaces: Adversarial Face Synthesis](https://mlsanity.com/paper/1908.05008/) (2019)
- [AnonymousNet: Natural Face De-Identification with Measurable Privacy](https://mlsanity.com/paper/1904.12620/) (2019)
- [Defeating Image Obfuscation with Deep Learning](https://mlsanity.com/paper/1609.00408/) (2016)
- [Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective](https://mlsanity.com/paper/1703.09471/) (2017)
- [PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation](https://mlsanity.com/paper/2108.10710/) (2021)
- [On adversarial patches: real-world attack on ArcFace-100 face recognition system](https://mlsanity.com/paper/1910.07067/) (2019)
- [PrivacyNet: Semi-Adversarial Networks for Multi-attribute Face Privacy](https://mlsanity.com/paper/2001.00561/) (2020)
- [Context-Aware Generative Adversarial Privacy](https://mlsanity.com/paper/1710.09549/) (2017)
- [Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Images](https://mlsanity.com/paper/1712.01066/) (2017)
- [Eye In-Painting with Exemplar Generative Adversarial Networks](https://mlsanity.com/paper/1712.03999/) (2017)
- [Privacy-Protective-GAN for Face De-identification](https://mlsanity.com/paper/1806.08906/) (2018)
- [It GAN DO Better: GAN-based Detection of Objects on Images with Varying Quality](https://mlsanity.com/paper/1912.01707/) (2019)
- [Paired 3D Model Generation with Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1808.03082/) (2018)

## Built on by (most cited first, 2 of 2)

- [SecureT2I: No More Unauthorized Manipulation on AI Generated Images from Prompts](https://mlsanity.com/paper/2507.03636/) (2025)
- [Identity masking effectiveness and gesture recognition: Effects of eye enhancement in seeing through the mask](https://mlsanity.com/paper/2301.08408/) (2023)

## Abstract (arXiv)

The use of social media websites and applications has become very popular and people share their photos on these networks. Automatic recognition and tagging of people's photos on these networks has raised privacy preservation issues and users seek methods for hiding their identities from these algorithms. Generative adversarial networks (GANs) are shown to be very powerful in generating face images in high diversity and also in editing face images. In this paper, we propose a Generative Mask-guided Face Image Manipulation (GMFIM) model based on GANs to apply imperceptible editing to the input face image to preserve the privacy of the person in the image. Our model consists of three main components: a) the face mask module to cut the face area out of the input image and omit the background, b) the GAN-based optimization module for manipulating the face image and hiding the identity and, c) the merge module for combining the background of the input image and the manipulated de-identified face image. Different criteria are considered in the loss function of the optimization step to produce high-quality images that are as similar as possible to the input image while they cannot be recognized by AFR systems. The results of the experiments on different datasets show that our model can achieve better performance against automated face recognition systems in comparison to the state-of-the-art methods and it catches a higher attack success rate in most experiments from a total of 18. Moreover, the generated images of our proposed model have the highest quality and are more pleasing to human eyes.