mlsanityData to 14 Oct 2026

Can Stochastic Gradient Langevin Dynamics Provide Differential Privacy for Deep Learning?

arXiv 2110.05057 ↗First posted 11 Oct 2021cs.LG cs.CRcited by 2

Not on the map. Why?

Abstract (arXiv)

Bayesian learning via Stochastic Gradient Langevin Dynamics (SGLD) has been suggested for differentially private learning. While previous research provides differential privacy bounds for SGLD at the initial steps of the algorithm or when close to convergence, the question of what differential privacy guarantees can be made in between remains unanswered. This interim region is of great importance, especially for Bayesian neural networks, as it is hard to guarantee convergence to the posterior. This paper shows that using SGLD might result in unbounded privacy loss for this interim region, even when sampling from the posterior is as differentially private as desired.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 50 references that are in the map. First posted on arXiv 11 Oct 2021.

Builds on

28 of 50 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2016Deep Learning with Differential Privacycited by 2,728
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  7. 2017Renyi Differential Privacycited by 650
  8. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 383
  9. 2013The Composition Theorem for Differential Privacycited by 297
  10. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 243
  11. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 218
  12. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 208
16 more
  1. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  2. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 199
  3. 2017Underdamped Langevin MCMC: A non-asymptotic analysiscited by 133
  4. 2017Convergence of Langevin MCMC in KL-divergencecited by 88
  5. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 73
  6. 2018Sampling Can Be Faster Than Optimizationcited by 68
  7. 2015Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlocited by 63
  8. 2018Detecting Violations of Differential Privacycited by 39
  9. 2016On the Theory and Practice of Privacy-Preserving Bayesian Data Analysiscited by 35
  10. 2020Faster Differentially Private Samplers via R'enyi Divergence Analysis of Discretized Langevin MCMCcited by 27
  11. 2017R'enyi Differential Privacy Mechanisms for Posterior Samplingcited by 16
  12. 2017On Connecting Stochastic Gradient MCMC and Differential Privacycited by 16
  13. 2020Wideband Channel Estimation with A Generative Adversarial Networkcited by 16
  14. 2019Characterizing Membership Privacy in Stochastic Gradient Langevin Dynamicscited by 8
  15. 2021Can Single-Shuffle SGD be Better than Reshuffling SGD and GD?cited by 8
  16. 2015On the Differential Privacy of Bayesian Inferencecited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2.

2 papers.

  1. 2023Multi-Resolution Diffusion for Privacy-Sensitive Recommender Systemscited by 4
  2. 2023DP-Fast MH: Private, Fast, and Accurate Metropolis-Hastings for Large-Scale Bayesian Inferencecited by 3