mlsanityData to 7 Oct 2026

Deep Learning with Differential Privacy

arXiv 1607.00133 ↗First posted 1 Jul 2016stat.ML cs.CR cs.LGcited by 2,682

Not on the map. Why?

Cited by 276 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Machine learning techniques based on neural networks are achieving remarkable results in a wide variety of domains. Often, the training of models requires large, representative datasets, which may be crowdsourced and contain sensitive information. The models should not expose private information in these datasets. Addressing this goal, we develop new algorithmic techniques for learning and a refined analysis of privacy costs within the framework of differential privacy. Our implementation and experiments demonstrate that we can train deep neural networks with non-convex objectives, under a modest privacy budget, and at a manageable cost in software complexity, training efficiency, and model quality.

Key facts. Cited by 2,682 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jul 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 343; 2022: 464; 2023: 513; 2024: 482; 2025: 473; 2026: 406.

2,682 papers.

Sort:

By area:

  1. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  3. 2022Red Teaming Language Models with Language Modelscited by 983
  4. 2021Ethical and social risks of harm from Language Modelscited by 957
  5. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  6. 2021Membership Inference Attacks From First Principlescited by 839
  7. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  8. 2023Extracting Training Data from Diffusion Modelscited by 723
  9. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  10. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  11. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  12. 2021Differentially Private Fine-tuning of Language Modelscited by 392
38 more
  1. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  2. 2021A Field Guide to Federated Optimizationcited by 312
  3. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  4. 2022A Survey of Machine Unlearningcited by 298
  5. 2021Unsolved Problems in ML Safetycited by 291
  6. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  7. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearningcited by 274
  8. 2023Towards Unbounded Machine Unlearningcited by 261
  9. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  10. 2022Transformers in Medical Imaging: A Surveycited by 245
  11. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  12. 2021Machine Unlearning of Features and Labelscited by 223
  13. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  14. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  15. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 211
  16. 2021Fast Yet Effective Machine Unlearningcited by 209
  17. 2021Trustworthy AI: From Principles to Practicescited by 206
  18. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  19. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 195
  20. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  21. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  22. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  23. 2021Unlearnable Examples: Making Personal Data Unexploitablecited by 188
  24. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  25. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  26. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  27. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  28. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  29. 2021Numerical Composition of Differential Privacycited by 174
  30. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  31. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  32. 2024Improving LoRA in Privacy-preserving Federated Learningcited by 158
  33. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  34. 2023Emergent and Predictable Memorization in Large Language Modelscited by 147
  35. 2022Reconstructing Training Data with Informed Adversariescited by 143
  36. 2021Federated Learning for Open Bankingcited by 138
  37. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 137
  38. 2023Are Diffusion Models Vulnerable to Membership Inference Attacks?cited by 135

50 most cited of 2,682 All 2,682 (JSON)