# Federated Stochastic Primal-dual Learning with Differential Privacy

arXiv: https://arxiv.org/abs/2204.12284 · HTML: https://mlsanity.com/paper/2204.12284/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 26 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning](https://mlsanity.com/paper/1812.00535/) (2018)
- [Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning](https://mlsanity.com/paper/2101.11203/) (2021)
- [FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data](https://mlsanity.com/paper/2005.11418/) (2020)
- [LDP-Fed: Federated Learning with Local Differential Privacy](https://mlsanity.com/paper/2006.03637/) (2020)
- [Federated Learning with Bayesian Differential Privacy](https://mlsanity.com/paper/1911.10071/) (2019)
- [On the Linear Convergence of the ADMM in Decentralized Consensus Optimization](https://mlsanity.com/paper/1307.5561/) (2013)
- [On Safeguarding Privacy and Security in the Framework of Federated Learning](https://mlsanity.com/paper/1909.06512/) (2019)
- [DP-ADMM: ADMM-based Distributed Learning with Differential Privacy](https://mlsanity.com/paper/1808.10101/) (2018)
- [Quantized Federated Learning under Transmission Delay and Outage Constraints](https://mlsanity.com/paper/2106.09397/) (2021)
- [No Peek: A Survey of private distributed deep learning](https://mlsanity.com/paper/1812.03288/) (2018)
- [Improving the Privacy and Accuracy of ADMM-Based Distributed Algorithms](https://mlsanity.com/paper/1806.02246/) (2018)
- [Convergence Rate of Distributed ADMM over Networks](https://mlsanity.com/paper/1601.00194/) (2016)
- [Recycled ADMM: Improve Privacy and Accuracy with Less Computation in Distributed Algorithms](https://mlsanity.com/paper/1810.03197/) (2018)
- [Privacy-preserving Distributed Machine Learning via Local Randomization and ADMM Perturbation](https://mlsanity.com/paper/1908.01059/) (2019)
- [Stochastic Proximal Gradient Consensus Over Random Networks](https://mlsanity.com/paper/1511.08905/) (2015)

## Built on by (most cited first, 5 of 5)

- [Differentially Private Federated Clustering over Non-IID Data](https://mlsanity.com/paper/2301.00955/) (2023)
- [Sparse Decentralized Federated Learning](https://mlsanity.com/paper/2308.16671/) (2023)
- [Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification](https://mlsanity.com/paper/2310.19558/) (2023)
- [Exact Penalty Method for Federated Learning](https://mlsanity.com/paper/2208.11231/) (2022)
- [Decentralized Federated Learning by Partial Message Exchange](https://mlsanity.com/paper/2603.01730/) (2026)

## Abstract (arXiv)

Federated learning (FL) is a new paradigm that enables many clients to jointly train a machine learning (ML) model under the orchestration of a parameter server while keeping the local data not being exposed to any third party. However, the training of FL is an interactive process between local clients and the parameter server. Such process would cause privacy leakage since adversaries may retrieve sensitive information by analyzing the overheard messages. In this paper, we propose a new federated stochastic primal-dual algorithm with differential privacy (FedSPD-DP). Compared to the existing methods, the proposed FedSPD-DP incorporates local stochastic gradient descent (local SGD) and partial client participation (PCP) for addressing the issues of communication efficiency and straggler effects due to randomly accessed clients. Our analysis shows that the data sampling strategy and PCP can enhance the data privacy whereas the larger number of local SGD steps could increase privacy leakage, revealing a non-trivial tradeoff between algorithm communication efficiency and privacy protection. Specifically, we show that, by guaranteeing (ε, δ)-DP for each client per communication round, the proposed algorithm guarantees (O(qε √p T), δ)-DP after T communication rounds while maintaining an O(1/√pTQ) convergence rate for a convex and non-smooth learning problem, where Q is the number of local SGD steps, p is the client sampling probability, q=maxi qi/√1-qi and qi is the data sampling probability of each client under PCP. Experiment results are presented to evaluate the practical performance of the proposed algorithm and comparison with state-of-the-art methods.