mlsanityData to 7 Oct 2026

Bounding Training Data Reconstruction in DP-SGD

arXiv 2302.07225 ↗First posted 14 Feb 2023cs.CR cs.LGcited by 42

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Differentially private training offers a protection which is usually interpreted as a guarantee against membership inference attacks. By proxy, this guarantee extends to other threats like reconstruction attacks attempting to extract complete training examples. Recent works provide evidence that if one does not need to protect against membership attacks but instead only wants to protect against training data reconstruction, then utility of private models can be improved because less noise is required to protect against these more ambitious attacks. We investigate this further in the context of DP-SGD, a standard algorithm for private deep learning, and provide an upper bound on the success of any reconstruction attack against DP-SGD together with an attack that empirically matches the predictions of our bound. Together, these two results open the door to fine-grained investigations on how to set the privacy parameters of DP-SGD in practice to protect against reconstruction attacks. Finally, we use our methods to demonstrate that different settings of the DP-SGD parameters leading to the same DP guarantees can result in significantly different success rates for reconstruction, indicating that the DP guarantee alone might not be a good proxy for controlling the protection against reconstruction attacks.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 42 of its 60 references that are in the map. First posted on arXiv 14 Feb 2023.

Builds on

42 of 60 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Optimal Membership Inference Bounds for Adaptive Composition of Sampled Gaussian Mechanisms12 Apr
  2. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks8 Mar
  3. 2021Membership Inference Attacks From First Principles7 Dec

Other references · 39

  1. 2016Wide Residual Networkscited by 3,057
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2019Deep Leakage from Gradientscited by 1,004
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2023Extracting Training Data from Diffusion Modelscited by 723
  8. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  9. 2017Renyi Differential Privacycited by 643
  10. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  11. 2020iDLG: Improved Deep Leakage from Gradientscited by 367
  12. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 320
27 more
  1. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  2. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  3. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
  4. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  5. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  6. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  7. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  8. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  9. 2021Numerical Composition of Differential Privacycited by 174
  10. 2022Reconstructing Training Data with Informed Adversariescited by 143
  11. 2022Reconstructing Training Data from Trained Neural Networkscited by 129
  12. 2018Protection Against Reconstruction and Its Applications in Private Federated Learningcited by 127
  13. 2021Gradient Inversion with Generative Image Priorcited by 101
  14. 2021Optimal Accounting of Differential Privacy via Characteristic Functioncited by 83
  15. 2021CAFE: Catastrophic Data Leakage in Vertical Federated Learningcited by 76
  16. 2019Overlearning Reveals Sensitive Attributescited by 74
  17. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
  18. 2020Individual Privacy Accounting via a Renyi Filtercited by 57
  19. 2022Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)cited by 45
  20. 2022Bounding Training Data Reconstruction in Private (Deep) Learningcited by 43
  21. 2022Large Scale Transfer Learning for Differentially Private Image Classificationcited by 40
  22. 2022Debugging Differential Privacy: A Case Study for Privacy Auditingcited by 39
  23. 2022Defending against Reconstruction Attacks with R'enyi Differential Privacycited by 30
  24. 2017Accuracy First: Selecting a Differential Privacy Level for Accuracy-Constrained ERMcited by 20
  25. 2022Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fanocited by 16
  26. 2022Fine-Tuning with Differential Privacy Necessitates an Additional Hyperparameter Searchcited by 15
  27. 2021Privately Publishable Per-instance Privacycited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 14; 2025: 14; 2026: 6.

42 papers.

Sort:

By area:

  1. 2024Auditing f-Differential Privacy in One Runcited by 28
  2. 2023Unleashing the Power of Randomization in Auditing Differentially Private MLcited by 19
  3. 2023Unlocking Accuracy and Fairness in Differentially Private Image Classificationcited by 16
  4. 2023Bounding data reconstruction attacks with the hypothesis testing interpretation of differential privacycited by 14
  5. 2023The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasetscited by 14
  6. 2023Graphical vs. Deep Generative Models: Measuring the Impact of Differentially Private Mechanisms and Budgets on Utilitycited by 13
  7. 2024Why Does Differential Privacy with Large Epsilon Defend Against Practical Membership Inference Attacks?cited by 12
  8. 2025Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacycited by 11
  9. 2024Improving Robustness to Model Inversion Attacks via Sparse Coding Architecturescited by 8
  10. 2024Beyond the Calibration Point: Mechanism Comparison in Differential Privacycited by 8
  11. 2024Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without Replacementcited by 7
  12. 2023Understanding Deep Gradient Leakage via Inversion Influence Functionscited by 6
30 more
  1. 2024Attack-Aware Noise Calibration for Differential Privacycited by 6
  2. 2025Gaussian DP for Reporting Differential Privacy Guarantees in Machine Learningcited by 6
  3. 2024Data Reconstruction Attacks and Defenses: A Systematic Evaluationcited by 5
  4. 2024Private Collaborative Edge Inference via Over-the-Air Computationcited by 4
  5. 2024Analyzing Inference Privacy Risks Through Gradients in Machine Learningcited by 4
  6. 2024Training Data Reconstruction: Privacy due to Uncertainty?cited by 4
  7. 2025The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditingcited by 4
  8. 2024Defending Against Data Reconstruction Attacks in Federated Learning: An Information Theory Approachcited by 3
  9. 2025Empirical Privacy Variancecited by 3
  10. 2025Machine Learning with Privacy for Protected Attributescited by 2
  11. 2023SoK: Memorisation in machine learningcited by 1
  12. 2023Reconciling AI Performance and Data Reconstruction Resilience for Medical Imagingcited by 1
  13. 2024From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attackscited by 1
  14. 2024Visual Privacy Auditing with Diffusion Modelscited by 1
  15. 2025Empirical Calibration and Metric Differential Privacy in Language Modelscited by 1
  16. 2025FedMeNF: Privacy-Preserving Federated Meta-Learning for Neural Fieldscited by 1
  17. 2025AI-in-the-Loop: Privacy Preserving Real-Time Scam Detection and Conversational Scambaiting by Leveraging LLMs and Federated Learningcited by 1
  18. 2025Cocoon: A System Architecture for Differentially Private Training with Correlated Noisescited by 1
  19. 2026Your Privacy Depends on Others: Collusion Vulnerabilities in Individual Differential Privacycited by 1
  20. 2026SoK: Challenges in Tabular Membership Inference Attackscited by 1
  21. 2026Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspectivecited by 1
  22. 2024Understanding Data Reconstruction Leakage in Federated Learning from a Theoretical Perspectivecited by 0
  23. 2025Training Set Reconstruction from Differentially Private Forests: How Effective is DP?cited by 0
  24. 2025An Interactive Framework for Implementing Privacy-Preserving Federated Learning: Experiments on Large Language Modelscited by 0
  25. 2025Securing Transfer-Learned Networks with Reverse Homomorphic Encryptioncited by 0
  26. 2025On Reconstructing Training Data From Bayesian Posteriors and Trained Modelscited by 0
  27. 2025Private Frequency Estimation Via Residue Number Systemscited by 0
  28. 2026Differential Privacy of Gaussian Process Posterior Samplingcited by 0
  29. 2026Memorization Is Not Extraction: Tight Differential-Privacy Bounds and Audit Blind Spotscited by 0
  30. 2026A Sharp Transition in Data Reconstruction under Differential Privacycited by 0