mlsanityData to 14 Oct 2026

Dyn-D2P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee

arXiv 2505.06651 ↗First posted 10 May 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Most existing decentralized learning methods with differential privacy (DP) guarantee rely on constant gradient clipping bounds and fixed-level DP Gaussian noises for each node throughout the training process, leading to a significant accuracy degradation compared to non-private counterparts. In this paper, we propose a new Dynamic Differentially Private Decentralized learning approach (termed Dyn-D2P) tailored for general time-varying directed networks. Leveraging the Gaussian DP (GDP) framework for privacy accounting, Dyn-D2P dynamically adjusts gradient clipping bounds and noise levels based on gradient convergence. This proposed dynamic noise strategy enables us to enhance model accuracy while preserving the total privacy budget. Extensive experiments on benchmark datasets demonstrate the superiority of Dyn-D2P over its counterparts employing fixed-level noises, especially under strong privacy guarantees. Furthermore, we provide a provable utility bound for Dyn-D2P that establishes an explicit dependency on network-related parameters, with a scaling factor of 1/√n in terms of the number of nodes n up to a bias error term induced by gradient clipping. To our knowledge, this is the first model utility analysis for differentially private decentralized non-convex optimization with dynamic gradient clipping bounds and noise levels.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 10 May 2025.

Builds on

32 of 50 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2016Deep Learning with Differential Privacycited by 2,728
  5. 2019Deep Leakage from Gradientscited by 1,021
  6. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  7. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  8. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 246
  9. 2015Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decompositioncited by 243
  10. 2019Differentially Private Learning with Adaptive Clippingcited by 220
  11. 2017Asynchronous Decentralized Parallel Stochastic Gradient Descentcited by 159
  12. 2018Stochastic Gradient Push for Distributed Deep Learningcited by 152
20 more
  1. 2020Understanding Gradient Clipping in Private SGD: A Geometric Perspectivecited by 142
  2. 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 128
  3. 2019Deep Learning with Gaussian Differential Privacycited by 114
  4. 2019Decentralized Deep Learning with Arbitrary Communication Compressioncited by 111
  5. 2017Efficient Private ERM for Smooth Objectivescited by 61
  6. 2020On Differentially Private Stochastic Convex Optimization with Heavy-tailed Datacited by 51
  7. 2019Differentially Private Meta-Learningcited by 41
  8. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  9. 2021Dynamic Differential-Privacy Preserving SGDcited by 30
  10. 2019Efficient Privacy-Preserving Stochastic Nonconvex Optimizationcited by 27
  11. 2020Distributed Training of Deep Learning Models: A Taxonomic Perspectivecited by 25
  12. 2021Low-Latency Federated Learning over Wireless Channels with Differential Privacycited by 25
  13. 2019The Value of Collaboration in Convex Machine Learning with Differential Privacycited by 24
  14. 2021Gradient Leakage Attack Resilient Deep Learningcited by 18
  15. 2022Tailoring Gradient Methods for Differentially-Private Distributed Optimizationcited by 18
  16. 2019Asynchronous Federated Learning with Differential Privacy for Edge Intelligencecited by 16
  17. 2023Securing Distributed SGD against Gradient Leakage Threatscited by 14
  18. 2022Adap DP-FL: Differentially Private Federated Learning with Adaptive Noisecited by 13
  19. 2023Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compressioncited by 9
  20. 2018LEASGD: an Efficient and Privacy-Preserving Decentralized Algorithm for Distributed Learningcited by 6

Built on by

Not cited yet.