mlsanityData to 13 Oct 2026

Non-Asymptotic Analysis of Online Local Private Learning with SGD

arXiv 2507.07041 ↗First posted 9 Jul 2025stat.ME cs.LG stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Differentially Private Stochastic Gradient Descent (DP-SGD) has been widely used for solving optimization problems with privacy guarantees in machine learning and statistics. Despite this, a systematic non-asymptotic convergence analysis for DP-SGD, particularly in the context of online problems and local differential privacy (LDP) models, remains largely elusive. Existing non-asymptotic analyses have focused on non-private optimization methods, and hence are not applicable to privacy-preserving optimization problems. This work initiates the analysis to bridge this gap and opens the door to non-asymptotic convergence analysis of private optimization problems. A general framework is investigated for the online LDP model in stochastic optimization problems. We assume that sensitive information from individuals is collected sequentially and aim to estimate, in real-time, a static parameter that pertains to the population of interest. Most importantly, we conduct a comprehensive non-asymptotic convergence analysis of the proposed estimators in finite-sample situations, which gives their users practical guidelines regarding the effect of various hyperparameters, such as step size, parameter dimensions, and privacy budgets, on convergence rates. Our proposed estimators are validated in the theoretical and practical realms by rigorous mathematical derivations and carefully constructed numerical experiments.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 17 of its 42 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

17 of 42 references.

References · 17

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,273
  2. 2014RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Responsecited by 271
  3. 2019Differentially Private Learning with Adaptive Clippingcited by 220
  4. 2019SGD: General Analysis and Improved Ratescited by 206
  5. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 170
  6. 2017Collecting Telemetry Data Privatelycited by 149
  7. 2016Minimax Optimal Procedures for Locally Private Estimationcited by 106
  8. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 72
  9. 2017Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10.12cited by 42
  10. 2021Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variancecited by 35
  11. 2021Dynamic Differential-Privacy Preserving SGDcited by 30
  12. 2015Statistical consistency and asymptotic normality for high-dimensional robust M-estimatorscited by 27
5 more
  1. 2021Differentially private inference via noisy optimizationcited by 27
  2. 2022Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamicscited by 19
  3. 2019Context-Aware Local Differential Privacycited by 18
  4. 2021Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descentcited by 16
  5. 2023Local Differential Privacy in Graph Neural Networks: a Reconstruction Approachcited by 5

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Faster Learning under Relaxed Local Differential Privacycited by 0