# Attribute-preserving Face Dataset Anonymization via Latent Code Optimization

arXiv: https://arxiv.org/abs/2303.11296 · HTML: https://mlsanity.com/paper/2303.11296/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 49 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 30 of 32)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond](https://mlsanity.com/paper/1804.10938/) (2018)
- [General Facial Representation Learning in a Visual-Linguistic Manner](https://mlsanity.com/paper/2112.03109/) (2021)
- [DeepPrivacy: A Generative Adversarial Network for Face Anonymization](https://mlsanity.com/paper/1909.04538/) (2019)
- [Talk-to-Edit: Fine-Grained Facial Editing via Dialog](https://mlsanity.com/paper/2109.04425/) (2021)
- [CIAGAN: Conditional Identity Anonymization Generative Adversarial Networks](https://mlsanity.com/paper/2005.09544/) (2020)
- [Anycost GANs for Interactive Image Synthesis and Editing](https://mlsanity.com/paper/2103.03243/) (2021)
- [AnonymousNet: Natural Face De-Identification with Measurable Privacy](https://mlsanity.com/paper/1904.12620/) (2019)
- [WarpedGANSpace: Finding non-linear RBF paths in GAN latent space](https://mlsanity.com/paper/2109.13357/) (2021)
- [Defeating Image Obfuscation with Deep Learning](https://mlsanity.com/paper/1609.00408/) (2016)
- [Faceless Person Recognition; Privacy Implications in Social Media](https://mlsanity.com/paper/1607.08438/) (2016)
- [IdentityDP: Differential Private Identification Protection for Face Images](https://mlsanity.com/paper/2103.01745/) (2021)
- [PandA: Unsupervised Learning of Parts and Appearances in the Feature Maps of GANs](https://mlsanity.com/paper/2206.00048/) (2022)
- [StyleMask: Disentangling the Style Space of StyleGAN2 for Neural Face Reenactment](https://mlsanity.com/paper/2209.13375/) (2022)
- [SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis](https://mlsanity.com/paper/1808.02531/) (2018)
- [Privacy-Protective-GAN for Face De-identification](https://mlsanity.com/paper/1806.08906/) (2018)
- [Tensor Component Analysis for Interpreting the Latent Space of GANs](https://mlsanity.com/paper/2111.11736/) (2021)
- [CFA-Net: Controllable Face Anonymization Network with Identity Representation Manipulation](https://mlsanity.com/paper/2105.11137/) (2021)
- [ContraCLIP: Interpretable GAN generation driven by pairs of contrasting sentences](https://mlsanity.com/paper/2206.02104/) (2022)

## Built on by (most cited first, 30 of 43)

- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces](https://mlsanity.com/paper/2307.10797/) (2023)
- [Diff-Privacy: Diffusion-based Face Privacy Protection](https://mlsanity.com/paper/2309.05330/) (2023)
- [G2Face: High-Fidelity Reversible Face Anonymization via Generative and Geometric Priors](https://mlsanity.com/paper/2408.09458/) (2024)
- [Face Anonymization Made Simple](https://mlsanity.com/paper/2411.00762/) (2024)
- [DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactment](https://mlsanity.com/paper/2403.17217/) (2024)
- [Are CLIP features all you need for Universal Synthetic Image Origin Attribution?](https://mlsanity.com/paper/2408.09153/) (2024)
- [Improving Fairness using Vision-Language Driven Image Augmentation](https://mlsanity.com/paper/2311.01573/) (2023)
- [Face De-identification: State-of-the-art Methods and Comparative Studies](https://mlsanity.com/paper/2411.09863/) (2024)
- [Latent Diffusion Models for Attribute-Preserving Image Anonymization](https://mlsanity.com/paper/2403.14790/) (2024)
- [MM2Latent: Text-to-facial image generation and editing in GANs with multimodal assistance](https://mlsanity.com/paper/2409.11010/) (2024)
- [One-shot Neural Face Reenactment via Finding Directions in GAN's Latent Space](https://mlsanity.com/paper/2402.03553/) (2024)
- [Privacy-preserving Optics for Enhancing Protection in Face De-identification](https://mlsanity.com/paper/2404.00777/) (2024)
- [NullFace: Training-Free Localized Face Anonymization](https://mlsanity.com/paper/2503.08478/) (2025)
- [Fairer Analysis and Demographically Balanced Face Generation for Fairer Face Verification](https://mlsanity.com/paper/2412.03349/) (2024)
- [Now You See Me, Now You Don't: A Unified Framework for Expression Consistent Anonymization in Talking Head Videos](https://mlsanity.com/paper/2601.11635/) (2026)
- [Anonymization Prompt Learning for Facial Privacy-Preserving Text-to-Image Generation](https://mlsanity.com/paper/2405.16895/) (2024)
- [For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives](https://mlsanity.com/paper/2407.03268/) (2024)
- [A Key-Driven Framework for Identity-Preserving Face Anonymization](https://mlsanity.com/paper/2409.03434/) (2024)
- [Generative Medical Image Anonymization Based on Latent Code Projection and Optimization](https://mlsanity.com/paper/2501.09114/) (2025)
- [Differentially Private 2D Human Pose Estimation](https://mlsanity.com/paper/2504.10190/) (2025)
- [FaceAnonyMixer: Cancelable Faces via Identity Consistent Latent Space Mixing](https://mlsanity.com/paper/2508.05636/) (2025)
- [Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization](https://mlsanity.com/paper/2510.24213/) (2025)
- [Reverse Personalization](https://mlsanity.com/paper/2512.22984/) (2025)
- [PrivacyGAN: robust generative image privacy](https://mlsanity.com/paper/2310.12590/) (2023)
- [Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification](https://mlsanity.com/paper/2311.08786/) (2023)
- [Multi-Task Faces (MTF) Data Set: A Legally and Ethically Compliant Collection of Face Images for Various Classification Tasks](https://mlsanity.com/paper/2311.11882/) (2023)
- [VerA: Versatile Anonymization Applicable to Clinical Facial Photographs](https://mlsanity.com/paper/2312.02124/) (2023)
- [Toward Fairer Face Recognition Datasets](https://mlsanity.com/paper/2406.16592/) (2024)
- [Forbes: Face Obfuscation Rendering via Backpropagation Refinement Scheme](https://mlsanity.com/paper/2407.14170/) (2024)

## Abstract (arXiv)

This work addresses the problem of anonymizing the identity of faces in a dataset of images, such that the privacy of those depicted is not violated, while at the same time the dataset is useful for downstream task such as for training machine learning models. To the best of our knowledge, we are the first to explicitly address this issue and deal with two major drawbacks of the existing state-of-the-art approaches, namely that they (i) require the costly training of additional, purpose-trained neural networks, and/or (ii) fail to retain the facial attributes of the original images in the anonymized counterparts, the preservation of which is of paramount importance for their use in downstream tasks. We accordingly present a task-agnostic anonymization procedure that directly optimizes the images' latent representation in the latent space of a pre-trained GAN. By optimizing the latent codes directly, we ensure both that the identity is of a desired distance away from the original (with an identity obfuscation loss), whilst preserving the facial attributes (using a novel feature-matching loss in FaRL's deep feature space). We demonstrate through a series of both qualitative and quantitative experiments that our method is capable of anonymizing the identity of the images whilst -- crucially -- better-preserving the facial attributes. We make the code and the pre-trained models publicly available at: https://github.com/chi0tzp/FALCO.