# Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamics

arXiv: https://arxiv.org/abs/2201.11980 · HTML: https://mlsanity.com/paper/2201.11980/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 28 Jan 2022.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Differentially Private Learning Needs Better Features (or Much More Data)](https://mlsanity.com/paper/2011.11660/) (2020)
- [CrypTen: Secure Multi-Party Computation Meets Machine Learning](https://mlsanity.com/paper/2109.00984/) (2021)
- [FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning](https://mlsanity.com/paper/2004.02229/) (2020)
- [Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent](https://mlsanity.com/paper/2102.05855/) (2021)
- [ARIANN: Low-Interaction Privacy-Preserving Deep Learning via Function Secret Sharing](https://mlsanity.com/paper/2006.04593/) (2020)

## Built on by (most cited first, 19 of 19)

- [Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss](https://mlsanity.com/paper/2205.13710/) (2022)
- [Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning](https://mlsanity.com/paper/2401.10371/) (2024)
- [Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)](https://mlsanity.com/paper/2203.05363/) (2022)
- [Faster high-accuracy log-concave sampling via algorithmic warm starts](https://mlsanity.com/paper/2302.10249/) (2023)
- [Resolving the Mixing Time of the Langevin Algorithm to its Stationary Distribution for Log-Concave Sampling](https://mlsanity.com/paper/2210.08448/) (2022)
- [Certified Machine Unlearning via Noisy Stochastic Gradient Descent](https://mlsanity.com/paper/2403.17105/) (2024)
- [Privacy Loss of Noisy Stochastic Gradient Descent Might Converge Even for Non-Convex Losses](https://mlsanity.com/paper/2305.09903/) (2023)
- [On the Statistical Complexity of Estimation and Testing under Privacy Constraints](https://mlsanity.com/paper/2210.02215/) (2022)
- [Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacy](https://mlsanity.com/paper/2310.14661/) (2023)
- [An Improved Privacy and Utility Analysis of Differentially Private SGD with Bounded Domain and Smooth Losses](https://mlsanity.com/paper/2502.17772/) (2025)
- [Differentially Private Sampling from Rashomon Sets, and the Universality of Langevin Diffusion for Convex Optimization](https://mlsanity.com/paper/2204.01585/) (2022)
- [Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping](https://mlsanity.com/paper/2403.02051/) (2024)
- [Recycling Scraps: Improving Private Learning by Leveraging Intermediate Checkpoints](https://mlsanity.com/paper/2210.01864/) (2022)
- [Non-Asymptotic Analysis of Online Local Private Learning with SGD](https://mlsanity.com/paper/2507.07041/) (2025)
- [Differential privacy guarantees of Markov chain Monte Carlo algorithms](https://mlsanity.com/paper/2502.17150/) (2025)
- [High-Dimensional Privacy-Utility Dynamics of Noisy Stochastic Gradient Descent on Least Squares](https://mlsanity.com/paper/2510.16687/) (2025)
- [R'enyi Differential Privacy for Heavy-Tailed SDEs via Fractional Poincar'e Inequalities](https://mlsanity.com/paper/2511.15634/) (2025)
- [Certified Per-Instance Unlearning Using Individual Sensitivity Bounds](https://mlsanity.com/paper/2602.15602/) (2026)
- [Rapid mixing for Gibbs measures in Riemannian manifolds](https://mlsanity.com/paper/2606.13453/) (2026)

## Abstract (arXiv)

We analyse the privacy leakage of noisy stochastic gradient descent by modeling R'enyi divergence dynamics with Langevin diffusions. Inspired by recent work on non-stochastic algorithms, we derive similar desirable properties in the stochastic setting. In particular, we prove that the privacy loss converges exponentially fast for smooth and strongly convex objectives under constant step size, which is a significant improvement over previous DP-SGD analyses. We also extend our analysis to arbitrary sequences of varying step sizes and derive new utility bounds. Last, we propose an implementation and our experiments show the practical utility of our approach compared to classical DP-SGD libraries.