Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence
Not on the map. Why?
Abstract (arXiv)
Differentially private federated learning (DP-FL) is a promising technique for collaborative model training while ensuring provable privacy for clients. However, optimizing the tradeoff between privacy and accuracy remains a critical challenge. To our best knowledge, we propose the first DP-FL framework (namely UDP-FL), which universally harmonizes any randomization mechanism (e.g., an optimal one) with the Gaussian Moments Accountant (viz. DP-SGD) to significantly boost accuracy and convergence. Specifically, UDP-FL demonstrates enhanced model performance by mitigating the reliance on Gaussian noise. The key mediator variable in this transformation is the R'enyi Differential Privacy notion, which is carefully used to harmonize privacy budgets. We also propose an innovative method to theoretically analyze the convergence for DP-FL (including our UDP-FL ) based on mode connectivity analysis. Moreover, we evaluate our UDP-FL through extensive experiments benchmarked against state-of-the-art (SOTA) methods, demonstrating superior performance on both privacy guarantees and model performance. Notably, UDP-FL exhibits substantial resilience against different inference attacks, indicating a significant advance in safeguarding sensitive data in federated learning environments.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 56 of its 102 references that are in the map. First posted on arXiv 20 Jul 2024.
Builds on
56 of 102 references.
References · 56
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2016Deep Learning with Differential Privacycited by 2,682
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
- 2019Deep Leakage from Gradientscited by 1,004
- 2021Membership Inference Attacks From First Principlescited by 839
- 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
- 2017Renyi Differential Privacycited by 643
- 2018Local SGD Converges Fast and Communicates Littlecited by 595
38 more
- 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
- 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
- 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
- 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
- 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 426
- 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
- 2014The Loss Surfaces of Multilayer Networkscited by 278
- 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
- 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
- 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
- 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 163
- 2020Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustnesscited by 130
- 2018Protection Against Reconstruction and Its Applications in Private Federated Learningcited by 127
- 2021Optimal Accounting of Differential Privacy via Characteristic Functioncited by 83
- 2021CAFE: Catastrophic Data Leakage in Vertical Federated Learningcited by 76
- 2020Local and Central Differential Privacy for Robustness and Privacy in Federated Learningcited by 72
- 2022Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakagecited by 72
- 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
- 2023Federated Learning of Gboard Language Models with Differential Privacycited by 64
- 2019Federated Learning with Bayesian Differential Privacycited by 60
- 2020Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorizationcited by 58
- 2018Smooth Loss Functions for Deep Top-k Classificationcited by 46
- 2020Federated Model Distillation with Noise-Free Differential Privacycited by 40
- 2022On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyondcited by 37
- 2022Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queriescited by 36
- 2018Differentially Private Data Generative Modelscited by 33
- 2020A Secure Federated Learning Framework for 5G Networkscited by 32
- 2021Federated f-Differential Privacycited by 29
- 2022Stronger Privacy Amplification by Shuffling for R'enyi and Approximate Differential Privacycited by 29
- 2020FLAME: Differentially Private Federated Learning in the Shuffle Modelcited by 27
- 2019Privacy-Preserving Gradient Boosting Decision Treescited by 20
- 2018Using Mode Connectivity for Loss Landscape Analysiscited by 19
- 2021A Novel Attribute Reconstruction Attack in Federated Learningcited by 18
- 2021On the Renyi Differential Privacy of the Shuffle Modelcited by 16
- 2017R'enyi Differential Privacy Mechanisms for Posterior Samplingcited by 15
- 2022Edgeworth Accountant: An Analytical Approach to Differential Privacy Compositioncited by 14
- 2023Task-Agnostic Privacy-Preserving Representation Learning for Federated Learning Against Attribute Inference Attackscited by 13
- 2021DeSMP: Differential Privacy-exploited Stealthy Model Poisoning Attacks in Federated Learningcited by 11
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2024: 1.
1 paper.