# FairDeFace: Evaluating the Fairness and Adversarial Robustness of Face Obfuscation Methods

arXiv: https://arxiv.org/abs/2503.08731 · HTML: https://mlsanity.com/paper/2503.08731/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 75 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 30 of 36)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [RetinaFace: Single-stage Dense Face Localisation in the Wild](https://mlsanity.com/paper/1905.00641/) (2019)
- [DSFD: Dual Shot Face Detector](https://mlsanity.com/paper/1810.10220/) (2018)
- [DeepPrivacy: A Generative Adversarial Network for Face Anonymization](https://mlsanity.com/paper/1909.04538/) (2019)
- [LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition](https://mlsanity.com/paper/2101.07922/) (2021)
- [CIAGAN: Conditional Identity Anonymization Generative Adversarial Networks](https://mlsanity.com/paper/2005.09544/) (2020)
- [Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs](https://mlsanity.com/paper/1907.06724/) (2019)
- [Learning to Anonymize Faces for Privacy Preserving Action Detection](https://mlsanity.com/paper/1803.11556/) (2018)
- [Natural and Effective Obfuscation by Head Inpainting](https://mlsanity.com/paper/1711.09001/) (2017)
- [DeepPrivacy2: Towards Realistic Full-Body Anonymization](https://mlsanity.com/paper/2211.09454/) (2022)
- [AnonymousNet: Natural Face De-Identification with Measurable Privacy](https://mlsanity.com/paper/1904.12620/) (2019)
- [Live Face De-Identification in Video](https://mlsanity.com/paper/1911.08348/) (2019)
- [Defeating Image Obfuscation with Deep Learning](https://mlsanity.com/paper/1609.00408/) (2016)
- [SSH: Single Stage Headless Face Detector](https://mlsanity.com/paper/1708.03979/) (2017)
- [Face-Off: Adversarial Face Obfuscation](https://mlsanity.com/paper/2003.08861/) (2020)
- [Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency](https://mlsanity.com/paper/2105.08667/) (2021)
- [The four-fifths rule is not disparate impact: a woeful tale of epistemic trespassing in algorithmic fairness](https://mlsanity.com/paper/2202.09519/) (2022)
- [Perceptual Indistinguishability-Net (PI-Net): Facial Image Obfuscation with Manipulable Semantics](https://mlsanity.com/paper/2104.01753/) (2021)
- [FoggySight: A Scheme for Facial Lookup Privacy](https://mlsanity.com/paper/2012.08588/) (2020)
- [Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges](https://mlsanity.com/paper/2305.14080/) (2023)

## Abstract (arXiv)

The lack of a common platform and benchmark datasets for evaluating face obfuscation methods has been a challenge, with every method being tested using arbitrary experiments, datasets, and metrics. While prior work has demonstrated that face recognition systems exhibit bias against some demographic groups, there exists a substantial gap in our understanding regarding the fairness of face obfuscation methods. Providing fair face obfuscation methods can ensure equitable protection across diverse demographic groups, especially since they can be used to preserve the privacy of vulnerable populations. To address these gaps, this paper introduces a comprehensive framework, named FairDeFace, designed to assess the adversarial robustness and fairness of face obfuscation methods. The framework introduces a set of modules encompassing data benchmarks, face detection and recognition algorithms, adversarial models, utility detection models, and fairness metrics. FairDeFace serves as a versatile platform where any face obfuscation method can be integrated, allowing for rigorous testing and comparison with other state-of-the-art methods. In its current implementation, FairDeFace incorporates 6 attacks, and several privacy, utility and fairness metrics. Using FairDeFace, and by conducting more than 500 experiments, we evaluated and compared the adversarial robustness of seven face obfuscation methods. This extensive analysis led to many interesting findings both in terms of the degree of robustness of existing methods and their biases against some gender or racial groups. FairDeFace also uses visualization of focused areas for both obfuscation and verification attacks to show not only which areas are mostly changed in the obfuscation process for some demographics, but also why they failed through focus area comparison of obfuscation and verification.