mlsanityData to 7 Oct 2026

DPlis: Boosting Utility of Differentially Private Deep Learning via Randomized Smoothing

arXiv 2103.01496 ↗First posted 2 Mar 2021cs.LG cs.CR stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Deep learning techniques have achieved remarkable performance in wide-ranging tasks. However, when trained on privacy-sensitive datasets, the model parameters may expose private information in training data. Prior attempts for differentially private training, although offering rigorous privacy guarantees, lead to much lower model performance than the non-private ones. Besides, different runs of the same training algorithm produce models with large performance variance. To address these issues, we propose DPlis--Differentially Private Learning wIth Smoothing. The core idea of DPlis is to construct a smooth loss function that favors noise-resilient models lying in large flat regions of the loss landscape. We provide theoretical justification for the utility improvements of DPlis. Extensive experiments also demonstrate that DPlis can effectively boost model quality and training stability under a given privacy budget.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 39 of its 76 references that are in the map. First posted on arXiv 2 Mar 2021.

Builds on

39 of 76 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  7. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  8. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  9. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  10. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  11. 2017Renyi Differential Privacycited by 643
  12. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
27 more
  1. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  2. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  3. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  4. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  5. 2018Scalable Private Learning with PATEcited by 266
  6. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
  7. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
  8. 2019Differentially Private Learning with Adaptive Clippingcited by 218
  9. 2013Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifierscited by 128
  10. 2020Tempered Sigmoid Activations for Deep Learning with Differential Privacycited by 121
  11. 2019Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learningcited by 117
  12. 2018Understanding Membership Inferences on Well-Generalized Learning Modelscited by 83
  13. 2018Private Selection from Private Candidatescited by 75
  14. 2019Differentially Private Model Publishing for Deep Learningcited by 69
  15. 2017Efficient Private ERM for Smooth Objectivescited by 61
  16. 2020Improving Robustness to Model Inversion Attacks via Mutual Information Regularizationcited by 49
  17. 2020Differentially Private Language Models Benefit from Public Pre-trainingcited by 46
  18. 2018Concentrated Differentially Private Gradient Descent with Adaptive per-Iteration Privacy Budgetcited by 31
  19. 2019Adversarial Neural Network Inversion via Auxiliary Knowledge Alignmentcited by 30
  20. 2018SmoothOut: Smoothing Out Sharp Minima to Improve Generalization in Deep Learningcited by 25
  21. 2014Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometrycited by 24
  22. 2016Conversational Contextual Cues: The Case of Personalization and History for Response Rankingcited by 21
  23. 2019DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERMcited by 15
  24. 2019GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Modelscited by 10
  25. 2019RIGA: Covert and Robust White-Box Watermarking of Deep Neural Networkscited by 9
  26. 2020Differentially Private Deep Learning with Direct Feedback Alignmentcited by 7
  27. 2019Differentially Private Regression and Classification with Sparse Gaussian Processescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 3; 2024: 2; 2025: 2; 2026: 1.

10 papers.

  1. 2021Selective Differential Privacy for Language Modelingcited by 61
  2. 2022Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregationcited by 50
  3. 2023Differentially Private Sharpness-Aware Trainingcited by 14
  4. 2024DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reductioncited by 10
  5. 2023Differentially Private Video Activity Recognitioncited by 9
  6. 2024Delving into Differentially Private Transformercited by 5
  7. 2023AIJack: Let's Hijack AI! Security and Privacy Risk Simulator for Machine Learningcited by 0
  8. 2025Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)cited by 0
  9. 2025Optimizer Dynamics at the Edge of Stability with Differential Privacycited by 0
  10. 2026Enhancing Differentially Private Mechanisms via Empirical Bayescited by 0