mlsanityData to 7 Oct 2026

Adaptive Hybrid Masking Strategy for Privacy-Preserving Face Recognition Against Model Inversion Attack

arXiv 2403.10558 ↗First posted 14 Mar 2024cs.CV cs.CR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The utilization of personal sensitive data in training face recognition (FR) models poses significant privacy concerns, as adversaries can employ model inversion attacks (MIA) to infer the original training data. Existing defense methods, such as data augmentation and differential privacy, have been employed to mitigate this issue. However, these methods often fail to strike an optimal balance between privacy and accuracy. To address this limitation, this paper introduces an adaptive hybrid masking algorithm against MIA. Specifically, face images are masked in the frequency domain using an adaptive MixUp strategy. Unlike the traditional MixUp algorithm, which is predominantly used for data augmentation, our modified approach incorporates frequency domain mixing. Previous studies have shown that increasing the number of images mixed in MixUp can enhance privacy preservation but at the expense of reduced face recognition accuracy. To overcome this trade-off, we develop an enhanced adaptive MixUp strategy based on reinforcement learning, which enables us to mix a larger number of images while maintaining satisfactory recognition accuracy. To optimize privacy protection, we propose maximizing the reward function (i.e., the loss function of the FR system) during the training of the strategy network. While the loss function of the FR network is minimized in the phase of training the FR network. The strategy network and the face recognition network can be viewed as antagonistic entities in the training process, ultimately reaching a more balanced trade-off. Experimental results demonstrate that our proposed hybrid masking scheme outperforms existing defense algorithms in terms of privacy preservation and recognition accuracy against MIA.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 69 of its 106 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

69 of 106 references.

References · 69

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  8. 2017Squeeze-and-Excitation Networkscited by 4,763
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  12. 2014How transferable are features in deep neural networks?cited by 1,216
38 more
  1. 2019Deep Leakage from Gradientscited by 1,004
  2. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
  3. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  4. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  5. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  6. 2016MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognitioncited by 621
  7. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  8. 2014Learning Face Representation from Scratchcited by 451
  9. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  10. 2020iDLG: Improved Deep Leakage from Gradientscited by 367
  11. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  12. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 320
  13. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  14. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
  15. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
  16. 2020Learning in the Frequency Domaincited by 196
  17. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  18. 2017A Survey on Homomorphic Encryption Schemes: Theory and Implementationcited by 154
  19. 2019Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learningcited by 145
  20. 2015Accelerating Very Deep Convolutional Networks for Classification and Detectioncited by 109
  21. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 106
  22. 2017Compressed Video Action Recognitioncited by 102
  23. 2021Gradient Inversion with Generative Image Priorcited by 101
  24. 2020R-GAP: Recursive Gradient Attack on Privacycited by 95
  25. 2020InstaHide: Instance-hiding Schemes for Private Distributed Learningcited by 90
  26. 2022Variational Model Inversion Attackscited by 75
  27. 2020FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learningcited by 74
  28. 2020Local Differential Privacy based Federated Learning for Internet of Thingscited by 74
  29. 2022Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakagecited by 72
  30. 2020Knowledge-Enriched Distributional Model Inversion Attackscited by 65
  31. 2022Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attackscited by 58
  32. 2020DarkneTZ: Towards Model Privacy at the Edge using Trusted Execution Environmentscited by 57
  33. 2022SPAct: Self-supervised Privacy Preservation for Action Recognitioncited by 47
  34. 2018Deep Residual Learning in the JPEG Transform Domaincited by 44
  35. 2019QUOTIENT: Two-Party Secure Neural Network Training and Predictioncited by 41
  36. 2018Learnable Image Encryptioncited by 33
  37. 2020Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attackscited by 32
  38. 2020Privacy Preserving Face Recognition Utilizing Differential Privacycited by 30

50 most cited of 69 All 69 (JSON)

Built on by

Not cited yet.