# Privacy-preserving Optics for Enhancing Protection in Face De-identification

arXiv: https://arxiv.org/abs/2404.00777 · HTML: https://mlsanity.com/paper/2404.00777/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 52 references that are in the map.
- First posted on arXiv 31 Mar 2024.

## Builds on (most cited first, 30 of 32)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [Few-Shot Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1905.01723/) (2019)
- [Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis](https://mlsanity.com/paper/1903.05628/) (2019)
- [Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression](https://mlsanity.com/paper/1904.07399/) (2019)
- [DeepPrivacy: A Generative Adversarial Network for Face Anonymization](https://mlsanity.com/paper/1909.04538/) (2019)
- [Privacy-Preserving Human Activity Recognition from Extreme Low Resolution](https://mlsanity.com/paper/1604.03196/) (2016)
- [Attribute-preserving Face Dataset Anonymization via Latent Code Optimization](https://mlsanity.com/paper/2303.11296/) (2023)
- [Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Images](https://mlsanity.com/paper/1712.01066/) (2017)
- [Password-conditioned Anonymization and Deanonymization with Face Identity Transformers](https://mlsanity.com/paper/1911.11759/) (2019)
- [RiDDLE: Reversible and Diversified De-identification with Latent Encryptor](https://mlsanity.com/paper/2303.05171/) (2023)
- [Human Pose Estimation on Privacy-Preserving Low-Resolution Depth Images](https://mlsanity.com/paper/2007.08340/) (2020)
- [PrivHAR: Recognizing Human Actions From Privacy-preserving Lens](https://mlsanity.com/paper/2206.03891/) (2022)
- [Person Re-Identification without Identification via Event Anonymization](https://mlsanity.com/paper/2308.04402/) (2023)
- [Extreme Low Resolution Activity Recognition with Multi-Siamese Embedding Learning](https://mlsanity.com/paper/1708.00999/) (2017)
- [Privacy-Preserving Action Recognition via Motion Difference Quantization](https://mlsanity.com/paper/2208.02459/) (2022)
- [Visual Content Privacy Protection: A Survey](https://mlsanity.com/paper/2303.16552/) (2023)
- [InfoScrub: Towards Attribute Privacy by Targeted Obfuscation](https://mlsanity.com/paper/2005.10329/) (2020)

## Built on by (most cited first, 4 of 4)

- [Recoverable Anonymization for Pose Estimation: A Privacy-Enhancing Approach](https://mlsanity.com/paper/2409.02715/) (2024)
- [Differentially Private Integrated Decision Gradients (IDG-DP) for Radar-based Human Activity Recognition](https://mlsanity.com/paper/2411.02099/) (2024)
- [MambaStyle: Efficient StyleGAN Inversion for Real Image Editing with State-Space Models](https://mlsanity.com/paper/2505.15822/) (2025)
- [Face De-Identification: A Domain-Centric Survey from Capture to Processing](https://mlsanity.com/paper/2607.25926/) (2026)

## Abstract (arXiv)

The modern surge in camera usage alongside widespread computer vision technology applications poses significant privacy and security concerns. Current artificial intelligence (AI) technologies aid in recognizing relevant events and assisting in daily tasks in homes, offices, hospitals, etc. The need to access or process personal information for these purposes raises privacy concerns. While software-level solutions like face de-identification provide a good privacy/utility trade-off, they present vulnerabilities to sniffing attacks. In this paper, we propose a hardware-level face de-identification method to solve this vulnerability. Specifically, our approach first learns an optical encoder along with a regression model to obtain a face heatmap while hiding the face identity from the source image. We also propose an anonymization framework that generates a new face using the privacy-preserving image, face heatmap, and a reference face image from a public dataset as input. We validate our approach with extensive simulations and hardware experiments.