mlsanityData to 13 Oct 2026

R'enyi Differential Privacy for Heavy-Tailed SDEs via Fractional Poincar'e Inequalities

arXiv 2511.15634 ↗First posted 19 Nov 2025stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Characterizing the differential privacy (DP) of learning algorithms has become a major challenge in recent years. In parallel, many studies suggested investigating the behavior of stochastic gradient descent (SGD) with heavy-tailed noise, both as a model for modern deep learning models and to improve their performance. However, most DP bounds focus on light-tailed noise, where satisfactory guarantees have been obtained but the proposed techniques do not directly extend to the heavy-tailed setting. Recently, the first DP guarantees for heavy-tailed SGD were obtained. These results provide (0,δ)-DP guarantees without requiring gradient clipping. Despite casting new light on the link between DP and heavy-tailed algorithms, these results have a strong dependence on the number of parameters and cannot be extended to other DP notions like the well-established R'enyi differential privacy (RDP). In this work, we propose to address these limitations by deriving the first RDP guarantees for heavy-tailed SDEs, as well as their discretized counterparts. Our framework is based on new R'enyi flow computations and the use of well-established fractional Poincar'e inequalities. Under the assumption that such inequalities are satisfied, we obtain DP guarantees that have a much weaker dependence on the dimension compared to prior art.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 30 of its 64 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

30 of 64 references.

References · 30

  1. 2016Deep Learning with Differential Privacycited by 2,718
  2. 2017Renyi Differential Privacycited by 648
  3. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 343
  4. 2019A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networkscited by 213
  5. 2020Understanding Gradient Clipping in Private SGD: A Geometric Perspectivecited by 142
  6. 2016Concentrated Differential Privacycited by 141
  7. 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 128
  8. 2020The Heavy-Tail Phenomenon in SGDcited by 123
  9. 2018Privacy Amplification by Iterationcited by 98
  10. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 72
  11. 2019Differentially Private Model Publishing for Deep Learningcited by 71
  12. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
18 more
  1. 2024Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearningcited by 48
  2. 2022Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)cited by 46
  3. 2021Heavy Tails in SGD and Compressibility of Overparametrized Neural Networkscited by 44
  4. 2021Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variancecited by 35
  5. 2020Faster Differentially Private Samplers via R'enyi Divergence Analysis of Discretized Langevin MCMCcited by 27
  6. 2020Three Variants of Differential Privacy: Lossless Conversion and Applicationscited by 25
  7. 2022Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamicscited by 19
  8. 2023Algorithmic Stability of Heavy-Tailed SGD with General Loss Functionscited by 15
  9. 2020Differentially Private Accelerated Optimization Algorithmscited by 13
  10. 2022Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squarescited by 12
  11. 2023Privacy Loss of Noisy Stochastic Gradient Descent Might Converge Even for Non-Convex Lossescited by 10
  12. 2022Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descentcited by 9
  13. 2024Private Stochastic Convex Optimization with Heavy Tails: Near-Optimality from Simple Reductionscited by 6
  14. 2023Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGDcited by 5
  15. 2024Generalization Bounds for Heavy-Tailed SDEs through the Fractional Fokker-Planck Equationcited by 5
  16. 2024A Separation in Heavy-Tailed Sampling: Gaussian vs. Stable Oracles for Proximal Samplerscited by 5
  17. 2024Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clippingcited by 3
  18. 2025Heavy-Tailed Privacy: The Symmetric alpha-Stable Privacy Mechanismcited by 1

Built on by

Not cited yet.