mlsanityData to 7 Oct 2026

A General Approach to Adding Differential Privacy to Iterative Training Procedures

arXiv 1812.06210 ↗First posted 15 Dec 2018cs.LG stat.MLcited by 84

Not on the map. Why?

Abstract (arXiv)

In this work we address the practical challenges of training machine learning models on privacy-sensitive datasets by introducing a modular approach that minimizes changes to training algorithms, provides a variety of configuration strategies for the privacy mechanism, and then isolates and simplifies the critical logic that computes the final privacy guarantees. A key challenge is that training algorithms often require estimating many different quantities (vectors) from the same set of examples --- for example, gradients of different layers in a deep learning architecture, as well as metrics and batch normalization parameters. Each of these may have different properties like dimensionality, magnitude, and tolerance to noise. By extending previous work on the Moments Accountant for the subsampled Gaussian mechanism, we can provide privacy for such heterogeneous sets of vectors, while also structuring the approach to minimize software engineering challenges.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 18; 2023: 16; 2024: 15; 2025: 6; 2026: 1.

84 papers.

Sort:

By area:

  1. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  2. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  3. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  4. 2022Differentially Private Diffusion Modelscited by 118
  5. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  6. 2021Label Leakage and Protection in Two-party Split Learningcited by 87
  7. 2023Differentially Private Diffusion Models Generate Useful Synthetic Imagescited by 85
  8. 2021Learning with User-Level Privacycited by 72
  9. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
  10. 2022Exploring the Limits of Differentially Private Deep Learning with Group-wise Clippingcited by 52
  11. 2022Do Gradient Inversion Attacks Make Federated Learning Unsafe?cited by 44
  12. 2021Node-Level Differentially Private Graph Neural Networkscited by 43
38 more
  1. 2022Composition of Differential Privacy & Privacy Amplification by Subsamplingcited by 42
  2. 2021Enforcing fairness in private federated learning via the modified method of differential multiplierscited by 38
  3. 2021On the Convergence and Calibration of Deep Learning with Differential Privacycited by 36
  4. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  5. 2021Differentially Private Federated Bayesian Optimization with Distributed Explorationcited by 32
  6. 2021DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregationcited by 31
  7. 2021Quantum machine learning with differential privacycited by 30
  8. 2021Privacy Regularization: Joint Privacy-Utility Optimization in Language Modelscited by 29
  9. 2021An Efficient DP-SGD Mechanism for Large Scale NLP Modelscited by 27
  10. 2021Tight and Robust Private Mean Estimation with Few Userscited by 26
  11. 2021The Role of Adaptive Optimizers for Honest Private Hyperparameter Selectioncited by 25
  12. 2022Learning to Generate Image Embeddings with User-level Differential Privacycited by 23
  13. 2024Differentially Private Fine-Tuning of Diffusion Modelscited by 18
  14. 2021Investigating Trade-offs in Utility, Fairness and Differential Privacy in Neural Networkscited by 17
  15. 2021Wide Network Learning with Differential Privacycited by 17
  16. 2021Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGXcited by 14
  17. 2022Feature Space Hijacking Attacks against Differentially Private Split Learningcited by 14
  18. 2021Privacy and Trust Redefined in Federated Machine Learningcited by 13
  19. 2021FedLess: Secure and Scalable Federated Learning Using Serverless Computingcited by 13
  20. 2023Differentially Private SGD Without Clipping Bias: An Error-Feedback Approachcited by 13
  21. 2022The Impact of Differential Privacy on Group Disparity Mitigationcited by 11
  22. 2024A Huber Loss Minimization Approach to Mean Estimation under User-level Differential Privacycited by 11
  23. 2021Differentially Private Normalizing Flows for Privacy-Preserving Density Estimationcited by 10
  24. 2023Federated Recommendation with Additive Personalizationcited by 9
  25. 2023ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacycited by 9
  26. 2023DP-Mix: Mixup-based Data Augmentation for Differentially Private Learningcited by 9
  27. 2023Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Trainingcited by 7
  28. 2023On the accuracy and efficiency of group-wise clipping in differentially private optimizationcited by 7
  29. 2021Task-aware Privacy Preservation for Multi-dimensional Datacited by 5
  30. 2021On the privacy-utility trade-off in differentially private hierarchical text classificationcited by 4
  31. 2022Algorithms for bounding contribution for histogram estimation under user-level privacycited by 4
  32. 2023Holistic Survey of Privacy and Fairness in Machine Learningcited by 4
  33. 2024Differentially Private Clustered Federated Learningcited by 4
  34. 2021DP-XGBoost: Private Machine Learning at Scalecited by 3
  35. 2022Privacy-preserving Generative Framework Against Membership Inference Attackscited by 3
  36. 2022Exploring the Unfairness of DP-SGD Across Settingscited by 3
  37. 2022Privacy-Preserving Deep Learning Model for Covid-19 Disease Detectioncited by 3
  38. 2023TraVaG: Differentially Private Trace Variant Generation Using GANscited by 3

50 most cited of 84 All 84 (JSON)