Optimizing the Numbers of Queries and Replies in Federated Learning with Differential Privacy
Not on the map. Why?
Abstract (arXiv)
Federated learning (FL) empowers distributed clients to collaboratively train a shared machine learning model through exchanging parameter information. Despite the fact that FL can protect clients' raw data, malicious users can still crack original data with disclosed parameters. To amend this flaw, differential privacy (DP) is incorporated into FL clients to disturb original parameters, which however can significantly impair the accuracy of the trained model. In this work, we study a crucial question which has been vastly overlooked by existing works: what are the optimal numbers of queries and replies in FL with DP so that the final model accuracy is maximized. In FL, the parameter server (PS) needs to query participating clients for multiple global iterations to complete training. Each client responds a query from the PS by conducting a local iteration. Our work investigates how many times the PS should query clients and how many times each client should reply the PS. We investigate two most extensively used DP mechanisms (i.e., the Laplace mechanism and Gaussian mechanisms). Through conducting convergence rate analysis, we can determine the optimal numbers of queries and replies in FL with DP so that the final model accuracy can be maximized. Finally, extensive experiments are conducted with publicly available datasets: MNIST and FEMNIST, to verify our analysis and the results demonstrate that properly setting the numbers of queries and replies can significantly improve the final model accuracy in FL with DP.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 5 Jul 2021.
Builds on
33 of 46 references.
References · 33
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2016Deep Learning with Differential Privacycited by 2,682
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
- 2019Deep Leakage from Gradientscited by 1,004
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
- 2019Towards Federated Learning at Scale: System Designcited by 826
- 2018LEAF: A Benchmark for Federated Settingscited by 780
- 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
- 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
- 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
21 more
- 2020iDLG: Improved Deep Leakage from Gradientscited by 367
- 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
- 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
- 2020Threats to Federated Learning: A Surveycited by 171
- 2017Collecting Telemetry Data Privatelycited by 147
- 2020LDP-Fed: Federated Learning with Local Differential Privacycited by 129
- 2018Protection Against Reconstruction and Its Applications in Private Federated Learningcited by 127
- 2018SGD and Hogwild! Convergence Without the Bounded Gradients Assumptioncited by 94
- 2020Convergence Time Optimization for Federated Learning over Wireless Networkscited by 81
- 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
- 2020A Framework for Evaluating Gradient Leakage Attacks in Federated Learningcited by 63
- 2020Cost-Effective Federated Learning Designcited by 61
- 2019Federated Learning with Bayesian Differential Privacycited by 60
- 2020Wireless Federated Learning with Local Differential Privacycited by 50
- 2019Differential Privacy-enabled Federated Learning for Sensitive Health Datacited by 44
- 2019Differentially Private Meta-Learningcited by 40
- 2020FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selectioncited by 35
- 2018Improved asynchronous parallel optimization analysis for stochastic incremental methodscited by 30
- 2019The Value of Collaboration in Convex Machine Learning with Differential Privacycited by 24
- 2018Tight Dimension Independent Lower Bound on the Expected Convergence Rate for Diminishing Step Sizes in SGDcited by 18
- 2018Differentially-Private "Draw and Discard" Machine Learningcited by 7
Built on by
Citing papers per year: 2023: 1.
1 paper.