mlsanityData to 14 Oct 2026

Differentially Private Clustered Federated Learning

arXiv 2405.19272 ↗First posted 29 May 2024cs.LG cs.CR cs.DCcited by 4

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL), which is a decentralized machine learning (ML) approach, often incorporates differential privacy (DP) to provide rigorous data privacy guarantees. Previous works attempted to address high structured data heterogeneity in vanilla FL settings through clustering clients (a.k.a clustered FL), but these methods remain sensitive and prone to errors, further exacerbated by the DP noise. This vulnerability makes the previous methods inappropriate for differentially private FL (DPFL) settings with structured data heterogeneity. To address this gap, we propose an algorithm for differentially private clustered FL, which is robust to the DP noise in the system and identifies the underlying clients' clusters correctly. To this end, we propose to cluster clients based on both their model updates and training loss values. Furthermore, for clustering clients' model updates at the end of the first round, our proposed approach addresses the server's uncertainties by employing large batch sizes as well as Gaussian Mixture Models (GMM) to reduce the impact of DP and stochastic noise and avoid potential clustering errors. This idea is efficient especially in privacy-sensitive scenarios with more DP noise. We provide theoretical analysis to justify our approach and evaluate it across diverse data distributions and privacy budgets. Our experimental results show its effectiveness in addressing large structured data heterogeneity in DPFL.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 51 of its 78 references that are in the map. First posted on arXiv 29 May 2024.

Builds on

51 of 78 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2016Deep Learning with Differential Privacycited by 2,728
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  7. 2019Deep Leakage from Gradientscited by 1,021
  8. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  9. 2017Federated Multi-Task Learningcited by 672
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  11. 2017Renyi Differential Privacycited by 650
  12. 2020Personalized Federated Learning with Moreau Envelopescited by 618
38 more
  1. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  2. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  3. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  4. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  5. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  6. 2019Agnostic Federated Learningcited by 456
  7. 2019Fair Resource Allocation in Federated Learningcited by 394
  8. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 346
  9. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 334
  10. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  11. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  12. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 271
  13. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 246
  14. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
  15. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  16. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 204
  17. 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 197
  18. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  19. 2020A Survey of Privacy Attacks in Machine Learningcited by 146
  20. 2020The Discrete Gaussian for Differential Privacycited by 135
  21. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  22. 2016Minimax Optimal Procedures for Locally Private Estimationcited by 107
  23. 2023Low-Cost High-Power Membership Inference Attackscited by 107
  24. 2020Tilted Empirical Risk Minimizationcited by 106
  25. 2018A General Approach to Adding Differential Privacy to Iterative Training Procedurescited by 84
  26. 2018Private Selection from Private Candidatescited by 75
  27. 2020Local Differential Privacy based Federated Learning for Internet of Thingscited by 75
  28. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  29. 2020Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacycited by 61
  30. 2020Differentially Private and Fair Deep Learning: A Lagrangian Dual Approachcited by 53
  31. 2022Differential Privacy and Fairness in Decisions and Learning Tasks: A Surveycited by 47
  32. 2021FedSoft: Soft Clustered Federated Learning with Proximal Local Updatingcited by 46
  33. 2019Augment your batch: better training with larger batchescited by 37
  34. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  35. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  36. 2022Disparate Impact in Differential Privacy from Gradient Misalignmentcited by 31
  37. 2022Proportional Fairness in Federated Learningcited by 21
  38. 2020Membership Inference Attacks and Defenses in Classification Modelscited by 16

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learningcited by 9
  2. 2024Position: Cracking the Code of Cascading Disparity Towards Marginalized Communitiescited by 4
  3. 2025Differentially Private Federated Clustering with Random Rebalancingcited by 0
  4. 2026Encryption-Compatible Clustered Federated Learning via Distributed Expectation-Maximization over Metadatacited by 0