mlsanityData to 7 Oct 2026

DP-CSGP: Differentially Private Stochastic Gradient Push with Compressed Communication

arXiv 2512.13583 ↗First posted 15 Dec 2025cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a Differentially Private Stochastic Gradient Push with Compressed communication (termed DP-CSGP) for decentralized learning over directed graphs. Different from existing works, the proposed algorithm is designed to maintain high model utility while ensuring both rigorous differential privacy (DP) guarantees and efficient communication. For general non-convex and smooth objective functions, we show that the proposed algorithm achieves a tight utility bound of Oleft( √dlog left( frac1δ right)/(√nJε) right) (J and d are the number of local samples and the dimension of decision variables, respectively) with left(ε, δright)-DP guarantee for each node, matching that of decentralized counterparts with exact communication. Extensive experiments on benchmark tasks show that, under the same privacy budget, DP-CSGP achieves comparable model accuracy with significantly lower communication cost than existing decentralized counterparts with exact communication.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 45 of its 67 references that are in the map. First posted on arXiv 15 Dec 2025.

Builds on

45 of 67 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  5. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  6. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  7. 2018Sparsified SGD with Memorycited by 373
  8. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
  9. 2018The Convergence of Sparsified Gradient Methodscited by 228
  10. 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 218
  11. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  12. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 199
33 more
  1. 2017Asynchronous Decentralized Parallel Stochastic Gradient Descentcited by 159
  2. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  3. 2018Stochastic Gradient Push for Distributed Deep Learningcited by 152
  4. 2020Understanding Gradient Clipping in Private SGD: A Geometric Perspectivecited by 140
  5. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 131
  6. 2020LDP-Fed: Federated Learning with Local Differential Privacycited by 129
  7. 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 127
  8. 2019Distributed Learning with Compressed Gradient Differencescited by 110
  9. 2019Decentralized Deep Learning with Arbitrary Communication Compressioncited by 109
  10. 2020Acceleration for Compressed Gradient Descent in Distributed and Federated Optimizationcited by 90
  11. 2018Communication Compression for Decentralized Trainingcited by 87
  12. 2013Distributed optimization over time-varying directed graphscited by 77
  13. 2021MARINA: Faster Non-Convex Distributed Learning with Compressioncited by 77
  14. 2020On Differentially Private Stochastic Convex Optimization with Heavy-tailed Datacited by 51
  15. 2022BEER: Fast O(1/T) Rate for Decentralized Nonconvex Optimization with Communication Compressioncited by 41
  16. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  17. 2015Taming the Wild: A Unified Analysis of Hogwild!-Style Algorithmscited by 31
  18. 2020SQuARM-SGD: Communication-Efficient Momentum SGD for Decentralized Optimizationcited by 31
  19. 2019Efficient Privacy-Preserving Stochastic Nonconvex Optimizationcited by 27
  20. 2022Private Non-Convex Federated Learning Without a Trusted Servercited by 26
  21. 2020Distributed Training of Deep Learning Models: A Taxonomic Perspectivecited by 25
  22. 2019The Value of Collaboration in Convex Machine Learning with Differential Privacycited by 24
  23. 2021CANITA: Faster Rates for Distributed Convex Optimization with Communication Compressioncited by 23
  24. 2020Private and Communication-Efficient Edge Learning: A Sparse Differential Gaussian-Masking Distributed SGD Approachcited by 18
  25. 2022Tailoring Gradient Methods for Differentially-Private Distributed Optimizationcited by 18
  26. 2019Asynchronous Federated Learning with Differential Privacy for Edge Intelligencecited by 16
  27. 2021Faster Rates for Compressed Federated Learning with Client-Variance Reductioncited by 14
  28. 2023Securing Distributed SGD against Gradient Leakage Threatscited by 13
  29. 2023Compressed Decentralized Proximal Stochastic Gradient Method for Nonconvex Composite Problems with Heterogeneous Datacited by 11
  30. 2023CEDAS: A Compressed Decentralized Stochastic Gradient Method with Improved Convergencecited by 9
  31. 2018LEASGD: an Efficient and Privacy-Preserving Decentralized Algorithm for Distributed Learningcited by 6
  32. 2024PrivSGP-VR: Differentially Private Variance-Reduced Stochastic Gradient Push with Tight Utility Boundscited by 4
  33. 2023A Linearly Convergent Robust Compressed Push-Pull Method for Decentralized Optimizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Decentralized Stochastic Subgradient-type Methods with Communication Compression for Nonsmooth Nonconvex Optimizationcited by 0