# Practical Digital Disguises: Leveraging Face Swaps to Protect Patient Privacy

arXiv: https://arxiv.org/abs/2204.03559 · HTML: https://mlsanity.com/paper/2204.03559/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 25 of 25)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection](https://mlsanity.com/paper/2001.00179/) (2020)
- [S3FD: Single Shot Scale-invariant Face Detector](https://mlsanity.com/paper/1708.05237/) (2017)
- [DeepFaceLab: Integrated, flexible and extensible face-swapping framework](https://mlsanity.com/paper/2005.05535/) (2020)
- [Deep Learning for Deepfakes Creation and Detection: A Survey](https://mlsanity.com/paper/1909.11573/) (2019)
- [The Devil of Face Recognition is in the Noise](https://mlsanity.com/paper/1807.11649/) (2018)
- [Natural and Effective Obfuscation by Head Inpainting](https://mlsanity.com/paper/1711.09001/) (2017)
- [AnonymousNet: Natural Face De-Identification with Measurable Privacy](https://mlsanity.com/paper/1904.12620/) (2019)
- [Live Face De-Identification in Video](https://mlsanity.com/paper/1911.08348/) (2019)
- [Defeating Image Obfuscation with Deep Learning](https://mlsanity.com/paper/1609.00408/) (2016)
- [Faceless Person Recognition; Privacy Implications in Social Media](https://mlsanity.com/paper/1607.08438/) (2016)
- [Real-time Convolutional Neural Networks for Emotion and Gender Classification](https://mlsanity.com/paper/1710.07557/) (2017)
- [Beyond Frontal Faces: Improving Person Recognition Using Multiple Cues](https://mlsanity.com/paper/1501.05703/) (2015)
- [Deepfake detection: humans vs. machines](https://mlsanity.com/paper/2009.03155/) (2020)
- [Deepfakes for Medical Video De-Identification: Privacy Protection and Diagnostic Information Preservation](https://mlsanity.com/paper/2003.00813/) (2020)
- [Going Deeper Into Face Detection: A Survey](https://mlsanity.com/paper/2103.14983/) (2021)

## Built on by (most cited first, 2 of 2)

- [Introducing Explicit Gaze Constraints to Face Swapping](https://mlsanity.com/paper/2305.16138/) (2023)
- [Towards mitigating uncann(eye)ness in face swaps via gaze-centric loss terms](https://mlsanity.com/paper/2402.03188/) (2024)

## Abstract (arXiv)

With rapid advancements in image generation technology, face swapping for privacy protection has emerged as an active area of research. The ultimate benefit is improved access to video datasets, e.g. in healthcare settings. Recent literature has proposed deep network-based architectures to perform facial swaps and reported the associated reduction in facial recognition accuracy. However, there is not much reporting on how well these methods preserve the types of semantic information needed for the privatized videos to remain useful for their intended application. Our main contribution is a novel end-to-end face swapping pipeline for recorded videos of standardized assessments of autism symptoms in children. Through this design, we are the first to provide a methodology for assessing the privacy-utility trade-offs for the face swapping approach to patient privacy protection. Our methodology can show, for example, that current deep network based face swapping is bottle-necked by face detection in real world videos, and the extent to which gaze and expression information is preserved by face swaps relative to baseline privatization methods such as blurring.