mlsanityData to 7 Oct 2026

Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamics

arXiv 2201.11980 ↗First posted 28 Jan 2022stat.ML cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

We analyse the privacy leakage of noisy stochastic gradient descent by modeling R'enyi divergence dynamics with Langevin diffusions. Inspired by recent work on non-stochastic algorithms, we derive similar desirable properties in the stochastic setting. In particular, we prove that the privacy loss converges exponentially fast for smooth and strongly convex objectives under constant step size, which is a significant improvement over previous DP-SGD analyses. We also extend our analysis to arbitrary sequences of varying step sizes and derive new utility bounds. Last, we propose an implementation and our experiments show the practical utility of our approach compared to classical DP-SGD libraries.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 10 of its 25 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

10 of 25 references.

References · 10

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2017Renyi Differential Privacycited by 643
  5. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  6. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  7. 2021CrypTen: Secure Multi-Party Computation Meets Machine Learningcited by 131
  8. 2020FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learningcited by 74
  9. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 71
  10. 2020ARIANN: Low-Interaction Privacy-Preserving Deep Learning via Function Secret Sharingcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 3; 2024: 3; 2025: 5; 2026: 1.

18 papers.

Sort:

By area:

  1. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
  2. 2024Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearningcited by 48
  3. 2022Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)cited by 45
  4. 2023Faster high-accuracy log-concave sampling via algorithmic warm startscited by 44
  5. 2022Resolving the Mixing Time of the Langevin Algorithm to its Stationary Distribution for Log-Concave Samplingcited by 21
  6. 2024Certified Machine Unlearning via Noisy Stochastic Gradient Descentcited by 13
  7. 2023Privacy Loss of Noisy Stochastic Gradient Descent Might Converge Even for Non-Convex Lossescited by 10
  8. 2022On the Statistical Complexity of Estimation and Testing under Privacy Constraintscited by 6
  9. 2023Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacycited by 4
  10. 2025An Improved Privacy and Utility Analysis of Differentially Private SGD with Bounded Domain and Smooth Lossescited by 4
  11. 2022Differentially Private Sampling from Rashomon Sets, and the Universality of Langevin Diffusion for Convex Optimizationcited by 3
  12. 2024Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clippingcited by 3
6 more
  1. 2022Recycling Scraps: Improving Private Learning by Leveraging Intermediate Checkpointscited by 1
  2. 2025Differential privacy guarantees of Markov chain Monte Carlo algorithmscited by 0
  3. 2025Non-Asymptotic Analysis of Online Local Private Learning with SGDcited by 0
  4. 2025High-Dimensional Privacy-Utility Dynamics of Noisy Stochastic Gradient Descent on Least Squarescited by 0
  5. 2025R'enyi Differential Privacy for Heavy-Tailed SDEs via Fractional Poincar'e Inequalitiescited by 0
  6. 2026Certified Per-Instance Unlearning Using Individual Sensitivity Boundscited by 0