# On Connecting Stochastic Gradient MCMC and Differential Privacy

arXiv: https://arxiv.org/abs/1712.09097 · HTML: https://mlsanity.com/paper/1712.09097/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Dec 2017.

## Built on by (most cited first, 16 of 16)

- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [On the Convergence and Calibration of Deep Learning with Differential Privacy](https://mlsanity.com/paper/2106.07830/) (2021)
- [Stability Based Generalization Bounds for Exponential Family Langevin Dynamics](https://mlsanity.com/paper/2201.03064/) (2022)
- [Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability](https://mlsanity.com/paper/2107.08461/) (2021)
- [Differentially Private Deep Learning with ModelMix](https://mlsanity.com/paper/2210.03843/) (2022)
- [Statistic Selection and MCMC for Differentially Private Bayesian Estimation](https://mlsanity.com/paper/2203.13377/) (2022)
- [DP-Fast MH: Private, Fast, and Accurate Metropolis-Hastings for Large-Scale Bayesian Inference](https://mlsanity.com/paper/2303.06171/) (2023)
- [Differentially private training of neural networks with Langevin dynamics for calibrated predictive uncertainty](https://mlsanity.com/paper/2107.04296/) (2021)
- [Can Stochastic Gradient Langevin Dynamics Provide Differential Privacy for Deep Learning?](https://mlsanity.com/paper/2110.05057/) (2021)
- [Differentially private partitioned variational inference](https://mlsanity.com/paper/2209.11595/) (2022)
- [Locally Differentially Private Bayesian Inference](https://mlsanity.com/paper/2110.14426/) (2021)
- [Differentially private stochastic expectation propagation (DP-SEP)](https://mlsanity.com/paper/2111.13219/) (2021)
- [Differentially Private Online Bayesian Estimation With Adaptive Truncation](https://mlsanity.com/paper/2301.08202/) (2023)
- [Differentially Private Statistical Inference through β-Divergence One Posterior Sampling](https://mlsanity.com/paper/2307.05194/) (2023)
- [Differential privacy guarantees of Markov chain Monte Carlo algorithms](https://mlsanity.com/paper/2502.17150/) (2025)
- [On Inherent Privacy of Posterior Sampling: A Unified R'enyi-Divergence Framework](https://mlsanity.com/paper/2610.19573/) (2026)

## Abstract (arXiv)

Significant success has been realized recently on applying machine learning to real-world applications. There have also been corresponding concerns on the privacy of training data, which relates to data security and confidentiality issues. Differential privacy provides a principled and rigorous privacy guarantee on machine learning models. While it is common to design a model satisfying a required differential-privacy property by injecting noise, it is generally hard to balance the trade-off between privacy and utility. We show that stochastic gradient Markov chain Monte Carlo (SG-MCMC) -- a class of scalable Bayesian posterior sampling algorithms proposed recently -- satisfies strong differential privacy with carefully chosen step sizes. We develop theory on the performance of the proposed differentially-private SG-MCMC method. We conduct experiments to support our analysis and show that a standard SG-MCMC sampler without any modification (under a default setting) can reach state-of-the-art performance in terms of both privacy and utility on Bayesian learning.