# Privacy-Preserving Face Recognition with Learnable Privacy Budgets in Frequency Domain

arXiv: https://arxiv.org/abs/2207.07316 · HTML: https://mlsanity.com/paper/2207.07316/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 15 Jul 2022.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition](https://mlsanity.com/paper/2004.00288/) (2020)
- [High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks](https://mlsanity.com/paper/1905.13545/) (2019)
- [Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments](https://mlsanity.com/paper/1708.08197/) (2017)
- [Learning in the Frequency Domain](https://mlsanity.com/paper/2002.12416/) (2020)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [FSDR: Frequency Space Domain Randomization for Domain Generalization](https://mlsanity.com/paper/2103.02370/) (2021)
- [InstaHide: Instance-hiding Schemes for Private Distributed Learning](https://mlsanity.com/paper/2010.02772/) (2020)
- [Mis-classified Vector Guided Softmax Loss for Face Recognition](https://mlsanity.com/paper/1912.00833/) (2019)
- [Quantization Guided JPEG Artifact Correction](https://mlsanity.com/paper/2004.09320/) (2020)
- [Deep Residual Learning in the JPEG Transform Domain](https://mlsanity.com/paper/1812.11690/) (2018)
- [Privacy Preserving Face Recognition Utilizing Differential Privacy](https://mlsanity.com/paper/2005.10486/) (2020)
- [Is Private Learning Possible with Instance Encoding?](https://mlsanity.com/paper/2011.05315/) (2020)
- [Obfuscation of Images via Differential Privacy: From Facial Images to General Images](https://mlsanity.com/paper/2102.11072/) (2021)
- [Federated Face Recognition](https://mlsanity.com/paper/2105.02501/) (2021)

## Built on by (most cited first, 28 of 28)

- [Privacy-preserving Adversarial Facial Features](https://mlsanity.com/paper/2305.05391/) (2023)
- [Privacy-Preserving Face Recognition Using Trainable Feature Subtraction](https://mlsanity.com/paper/2403.12457/) (2024)
- [TransFace++: Rethinking the Face Recognition Paradigm with a Focus on Accuracy, Efficiency, and Security](https://mlsanity.com/paper/2308.10133/) (2023)
- [Privacy-Preserving Face Recognition Using Random Frequency Components](https://mlsanity.com/paper/2308.10461/) (2023)
- [SlerpFace: Face Template Protection via Spherical Linear Interpolation](https://mlsanity.com/paper/2407.03043/) (2024)
- [Clip Body and Tail Separately: High Probability Guarantees for DPSGD with Heavy Tails](https://mlsanity.com/paper/2405.17529/) (2024)
- [Privacy-Preserving Remote Heart Rate Estimation from Facial Videos](https://mlsanity.com/paper/2306.01141/) (2023)
- [DiffMI: Breaking Face Recognition Privacy via Diffusion-Driven Training-Free Model Inversion](https://mlsanity.com/paper/2504.18015/) (2025)
- [Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples](https://mlsanity.com/paper/2310.12707/) (2023)
- [PCDP-SGD: Improving the Convergence of Differentially Private SGD via Projection in Advance](https://mlsanity.com/paper/2312.03792/) (2023)
- [Privacy-Preserving Face Recognition in Hybrid Frequency-Color Domain](https://mlsanity.com/paper/2401.13386/) (2024)
- [PixelFade: Privacy-preserving Person Re-identification with Noise-guided Progressive Replacement](https://mlsanity.com/paper/2408.05543/) (2024)
- [CryptoFace: End-to-End Encrypted Face Recognition](https://mlsanity.com/paper/2509.00332/) (2025)
- [LDP-Slicing: Local Differential Privacy for Images via Randomized Bit-Plane Slicing](https://mlsanity.com/paper/2603.03711/) (2026)
- [Seeing is not Believing: An Identity Hider for Human Vision Privacy Protection](https://mlsanity.com/paper/2307.00481/) (2023)
- [A Key-Driven Framework for Identity-Preserving Face Anonymization](https://mlsanity.com/paper/2409.03434/) (2024)
- [Facial Expression Recognition with Controlled Privacy Preservation and Feature Compensation](https://mlsanity.com/paper/2412.00277/) (2024)
- [SemDP: Semantic-level Differential Privacy Protection for Face Datasets](https://mlsanity.com/paper/2412.15590/) (2024)
- [SASMU: boost the performance of generalized recognition model using synthetic face dataset](https://mlsanity.com/paper/2306.01449/) (2023)
- [Adaptive Hybrid Masking Strategy for Privacy-Preserving Face Recognition Against Model Inversion Attack](https://mlsanity.com/paper/2403.10558/) (2024)
- [Biometric Authentication Based on Enhanced Remote Photoplethysmography Signal Morphology](https://mlsanity.com/paper/2407.04127/) (2024)
- [Robust Skin Color Driven Privacy Preserving Face Recognition via Function Secret Sharing](https://mlsanity.com/paper/2407.05045/) (2024)
- [RDP: Ranked Differential Privacy for Facial Feature Protection in Multiscale Sparsified Subspace](https://mlsanity.com/paper/2408.00294/) (2024)
- [PPFL-RDSN: Privacy-Preserving Federated Learning-based Residual Dense Spatial Networks for Encrypted Lossy Image Reconstruction](https://mlsanity.com/paper/2507.00230/) (2025)
- [Realistic Face Reconstruction from Facial Embeddings via Diffusion Models](https://mlsanity.com/paper/2602.13168/) (2026)
- [Privatar: Scalable Privacy-preserving Multi-user VR via Secure Offloading](https://mlsanity.com/paper/2604.17476/) (2026)
- [Unveiling Deepfakes: A Frequency-Aware Triple Branch Network for Deepfake Detection](https://mlsanity.com/paper/2604.17477/) (2026)
- [DecoyFace: Beyond Obfuscation via Controllable and Imperceptible Identity Misdirection for Privacy-Preserving Face Recognition](https://mlsanity.com/paper/2607.17504/) (2026)

## Abstract (arXiv)

Face recognition technology has been used in many fields due to its high recognition accuracy, including the face unlocking of mobile devices, community access control systems, and city surveillance. As the current high accuracy is guaranteed by very deep network structures, facial images often need to be transmitted to third-party servers with high computational power for inference. However, facial images visually reveal the user's identity information. In this process, both untrusted service providers and malicious users can significantly increase the risk of a personal privacy breach. Current privacy-preserving approaches to face recognition are often accompanied by many side effects, such as a significant increase in inference time or a noticeable decrease in recognition accuracy. This paper proposes a privacy-preserving face recognition method using differential privacy in the frequency domain. Due to the utilization of differential privacy, it offers a guarantee of privacy in theory. Meanwhile, the loss of accuracy is very slight. This method first converts the original image to the frequency domain and removes the direct component termed DC. Then a privacy budget allocation method can be learned based on the loss of the back-end face recognition network within the differential privacy framework. Finally, it adds the corresponding noise to the frequency domain features. Our method performs very well with several classical face recognition test sets according to the extensive experiments.