mlsanityData to 7 Oct 2026

Convergent Differential Privacy Analysis for General Federated Learning

arXiv 2408.15621 ↗First posted 28 Aug 2024cs.LG cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

The powerful cooperation of federated learning (FL) and differential privacy~(DP) provides a promising paradigm for the large-scale private clients. However, existing analyses in FL-DP mostly rely on the composition theorem and cannot tightly quantify the privacy leakage challenges, which is tight for a few communication rounds but yields an arbitrarily loose and divergent bound eventually. This also implies a counterintuitive judgment, suggesting that FL-DP may not provide adequate privacy support during long-term training under constant-level noisy perturbations, yielding discrepancy between the theoretical and experimental results. To further investigate the convergent privacy and reliability of the FL-DP framework, in this paper, we comprehensively evaluate the worst privacy of two classical methods under the non-convex and smooth objectives based on the f-DP analysis. With the aid of the shifted interpolation technique, we successfully prove that privacy in ttfamily Noisy-FedAvg has a tight convergent bound. Moreover, with the regularization of the proxy term, privacy in ttfamily Noisy-FedProx has a stable constant lower bound. Our analysis further demonstrates a solid theoretical foundation for the reliability of privacy in FL-DP. Meanwhile, our conclusions can also be losslessly converted to other classical DP analytical frameworks, e.g. (ε,δ)-DP and Racuteenyi-DP~(RDP), to provide more fine-grained understandings for the FL-DP frameworks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 57 of its 97 references that are in the map. First posted on arXiv 28 Aug 2024.

Builds on

57 of 97 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  7. 2017Renyi Differential Privacycited by 643
  8. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  9. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  10. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
  11. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  12. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
38 more
  1. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  2. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  3. 2013The Composition Theorem for Differential Privacycited by 290
  4. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  5. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  6. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  7. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  8. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  9. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 103
  10. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  11. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  12. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  13. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  14. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  15. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 71
  16. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  17. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
  18. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  19. 2020Faster Non-Convex Federated Learning via Global and Local Momentumcited by 48
  20. 2019Local Differential Privacy for Deep Learningcited by 46
  21. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  22. 2022Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)cited by 45
  23. 2021STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learningcited by 43
  24. 2021Federated Learning with Local Differential Privacy: Trade-offs between Privacy, Utility, and Communicationcited by 38
  25. 2022Differentially Private Federated Learning with Local Regularization and Sparsificationcited by 35
  26. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
  27. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  28. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  29. 2022Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacycited by 32
  30. 2021Federated f-Differential Privacycited by 29
  31. 2022Private Non-Convex Federated Learning Without a Trusted Servercited by 26
  32. 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
  33. 2020Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacycited by 22
  34. 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
  35. 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
  36. 2021Decentralized Wireless Federated Learning with Differential Privacycited by 18
  37. 2023Visual Prompt Based Personalized Federated Learningcited by 16
  38. 2023Fusion of Global and Local Knowledge for Personalized Federated Learningcited by 14

50 most cited of 57 All 57 (JSON)

Built on by

Not cited yet.