mlsanityData to 14 Oct 2026

Statistical Inference for Differentially Private Stochastic Gradient Descent

arXiv 2507.20560 ↗First posted 28 Jul 2025stat.ML cs.LG stat.MEcited by 2

Not on the map. Why?

Abstract (arXiv)

Privacy preservation in machine learning, particularly through Differentially Private Stochastic Gradient Descent (DP-SGD), is critical for sensitive data analysis. However, existing statistical inference methods for SGD predominantly focus on cyclic subsampling, while DP-SGD requires randomized subsampling. This paper first bridges this gap by establishing the asymptotic properties of SGD under the randomized rule and extending these results to DP-SGD. For the output of DP-SGD, we show that the asymptotic variance decomposes into statistical, sampling, and privacy-induced components. Two methods are proposed for constructing valid confidence intervals: the plug-in method and the random scaling method. We also perform extensive numerical analysis, which shows that the proposed confidence intervals achieve nominal coverage rates while maintaining privacy.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 34 references that are in the map. First posted on arXiv 28 Jul 2025.

Builds on

19 of 34 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  3. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 383
  4. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 341
  5. 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 242
  6. 2020Stability of Stochastic Gradient Descent on Nonsmooth Convex Lossescited by 173
  7. 2020Understanding Gradient Clipping in Private SGD: A Geometric Perspectivecited by 142
  8. 2020The Heavy-Tail Phenomenon in SGDcited by 124
  9. 2019Deep Learning with Gaussian Differential Privacycited by 114
  10. 2023Revisiting Gradient Clipping: Stochastic bias and tight convergence guaranteescited by 83
  11. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 73
  12. 2016Statistical Inference for Model Parameters in Stochastic Gradient Descentcited by 71
7 more
  1. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 61
  2. 2015Differentially Private Ordinary Least Squarescited by 36
  3. 2021Fast and Robust Online Inference with Stochastic Gradient Descent via Random Scalingcited by 28
  4. 2021Differentially private inference via noisy optimizationcited by 27
  5. 2021Differentially Private SGD with Non-Smooth Lossescited by 22
  6. 2018HiGrad: Uncertainty Quantification for Online Learning and Stochastic Approximationcited by 13
  7. 2022Online Statistical Inference for Contextual Bandits via Stochastic Gradient Descentcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Statistical Inference on Gradient Flowscited by 1
  2. 2026Differentially Private Two-Stage Empirical Risk Minimization with Applications to Individualized Treatment Rulecited by 0