mlsanityData to 14 Oct 2026

Deep Learning with Data Privacy via Residual Perturbation

arXiv 2408.05723 ↗First posted 11 Aug 2024cs.LG cs.CR cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Protecting data privacy in deep learning (DL) is of crucial importance. Several celebrated privacy notions have been established and used for privacy-preserving DL. However, many existing mechanisms achieve privacy at the cost of significant utility degradation and computational overhead. In this paper, we propose a stochastic differential equation-based residual perturbation for privacy-preserving DL, which injects Gaussian noise into each residual mapping of ResNets. Theoretically, we prove that residual perturbation guarantees differential privacy (DP) and reduces the generalization gap of DL. Empirically, we show that residual perturbation is computationally efficient and outperforms the state-of-the-art differentially private stochastic gradient descent (DPSGD) in utility maintenance without sacrificing membership privacy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 73 references that are in the map. First posted on arXiv 11 Aug 2024.

Builds on

39 of 73 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2016Deep Learning with Differential Privacycited by 2,728
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  7. 2019Deep Leakage from Gradientscited by 1,021
  8. 2018Robustness May Be at Odds with Accuracycited by 880
  9. 2016Stealing Machine Learning Models via Prediction APIscited by 619
  10. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  11. 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
  12. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
27 more
  1. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  2. 2018Scalable Private Learning with PATEcited by 269
  3. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 246
  4. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 236
  5. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 218
  6. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 204
  7. 2017Generating Natural Adversarial Examplescited by 183
  8. 2018Disentangling Adversarial Robustness and Generalizationcited by 172
  9. 2019Are Labels Required for Improving Adversarial Robustness?cited by 163
  10. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 149
  11. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  12. 2018Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attackcited by 125
  13. 2019Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noisecited by 101
  14. 2018A General Approach to Adding Differential Privacy to Iterative Training Procedurescited by 84
  15. 2019Differentially Private Model Publishing for Deep Learningcited by 71
  16. 2017Detecting Cancer Metastases on Gigapixel Pathology Imagescited by 52
  17. 2019Robustness to Adversarial Perturbations in Learning from Incomplete Datacited by 49
  18. 2018Privacy-preserving Predictioncited by 48
  19. 2016You are Who You Know and How You Behave: Attribute Inference Attacks via Users' Social Friends and Behaviorscited by 34
  20. 2018ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuraciescited by 18
  21. 2019DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERMcited by 15
  22. 2019P3SGD: Patient Privacy Preserving SGD for Regularizing Deep CNNs in Pathological Image Classificationcited by 14
  23. 2019Privacy-Utility Trade-off of Linear Regression under Random Projections and Additive Noisecited by 13
  24. 2018D'ej`a Vu: an empirical evaluation of the memorization properties of ConvNetscited by 10
  25. 2019Characterizing Membership Privacy in Stochastic Gradient Langevin Dynamicscited by 8
  26. 2019AnomiGAN: Generative adversarial networks for anonymizing private medical datacited by 3
  27. 2018Model-Agnostic Private Learning via Stabilitycited by 2

Built on by

Not cited yet.