mlsanityData to 14 Oct 2026

Differentially Private Federated Learning on Heterogeneous Data

arXiv 2111.09278 ↗First posted 17 Nov 2021cs.LG cs.CR math.STcited by 74

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a paradigm for large-scale distributed learning which faces two key challenges: (i) efficient training from highly heterogeneous user data, and (ii) protecting the privacy of participating users. In this work, we propose a novel FL approach (DP-SCAFFOLD) to tackle these two challenges together by incorporating Differential Privacy (DP) constraints into the popular SCAFFOLD algorithm. We focus on the challenging setting where users communicate with a "honest-but-curious" server without any trusted intermediary, which requires to ensure privacy not only towards a third-party with access to the final model but also towards the server who observes all user communications. Using advanced results from DP theory, we establish the convergence of our algorithm for convex and non-convex objectives. Our analysis clearly highlights the privacy-utility trade-off under data heterogeneity, and demonstrates the superiority of DP-SCAFFOLD over the state-of-the-art algorithm DP-FedAvg when the number of local updates and the level of heterogeneity grow. Our numerical results confirm our analysis and show that DP-SCAFFOLD provides significant gains in practice.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Builds on 30 of its 53 references that are in the map. First posted on arXiv 17 Nov 2021.

Builds on

30 of 53 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2016Deep Learning with Differential Privacycited by 2,728
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  7. 2020Adaptive Federated Optimizationcited by 1,046
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  9. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  10. 2017Renyi Differential Privacycited by 650
  11. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  12. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
18 more
  1. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  2. 2021A Field Guide to Federated Optimizationcited by 315
  3. 2013The Composition Theorem for Differential Privacycited by 297
  4. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  5. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 236
  6. 2019Differentially Private Learning with Adaptive Clippingcited by 220
  7. 2019SGD: General Analysis and Improved Ratescited by 207
  8. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  9. 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 164
  10. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 154
  11. 2018Distributed Differential Privacy via Shufflingcited by 115
  12. 2016Minimax Optimal Procedures for Locally Private Estimationcited by 107
  13. 2020Local Differential Privacy based Federated Learning for Internet of Thingscited by 75
  14. 2019The Privacy Blanket of the Shuffle Modelcited by 74
  15. 2019Federated Learning with Bayesian Differential Privacycited by 60
  16. 2017Personalized and Private Peer-to-Peer Machine Learningcited by 54
  17. 2019Scalable and Differentially Private Distributed Aggregation in the Shuffled Modelcited by 20
  18. 2021Renyi Differential Privacy of the Subsampled Shuffle Model in Distributed Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 19; 2024: 23; 2025: 16; 2026: 9.

74 papers.

Sort:

By area:

  1. 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 53
  2. 2023Conformal Prediction for Federated Uncertainty Quantification Under Label Shiftcited by 29
  3. 2023One-Shot Federated Conformal Predictioncited by 27
  4. 2022Private Non-Convex Federated Learning Without a Trusted Servercited by 26
  5. 2023Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneitycited by 26
  6. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  7. 2022Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learningcited by 23
  8. 2024The Privacy Power of Correlated Noise in Decentralized Learningcited by 18
  9. 2022Adap DP-FL: Differentially Private Federated Learning with Adaptive Noisecited by 13
  10. 2024Revisiting Ensembling in One-Shot Federated Learningcited by 13
  11. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 13
  12. 2022Unraveling the Connections between Privacy and Certified Robustness in Federated Learning Against Poisoning Attackscited by 12
38 more
  1. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  2. 2023Compression with Exact Error Distribution for Federated Learningcited by 11
  3. 2023DIFF2: Differential Private Optimization via Gradient Differences for Nonconvex Distributed Learningcited by 10
  4. 2023Convergence and Privacy of Decentralized Nonconvex Optimization with Gradient Clipping and Communication Compressioncited by 9
  5. 2024Cross-silo Federated Learning with Record-level Personalized Differential Privacycited by 9
  6. 2024DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptationcited by 9
  7. 2024Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learningcited by 9
  8. 2024Private Heterogeneous Federated Learning Without a Trusted Server Revisited: Error-Optimal and Communication-Efficient Algorithms for Convex Lossescited by 8
  9. 2025Towards Trustworthy Federated Learning with Untrusted Participantscited by 8
  10. 2023Composite federated learning with heterogeneous datacited by 7
  11. 2023Theoretically Principled Federated Learning for Balancing Privacy and Utilitycited by 6
  12. 2023Mean Estimation Under Heterogeneous Privacy Demandscited by 6
  13. 2024The Effect of Quantization in Federated Learning: A R'enyi Differential Privacy Perspectivecited by 6
  14. 2023FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximationscited by 5
  15. 2023From Noisy Fixed-Point Iterations to Private ADMM for Centralized and Federated Learningcited by 5
  16. 2023Dynamic Privacy Allocation for Locally Differentially Private Federated Learning with Composite Objectivescited by 5
  17. 2024FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Datacited by 5
  18. 2024Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Datacited by 5
  19. 2025FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Healthcited by 5
  20. 2026Rethinking LoRA for Privacy-Preserving Federated Learning in Large Modelscited by 5
  21. 2024Differentially Private Clustered Federated Learningcited by 4
  22. 2024A survey on secure decentralized optimization and learningcited by 4
  23. 2024Efficient Federated Unlearning with Adaptive Differential Privacy Preservationcited by 4
  24. 2024Enhancing Privacy in Federated Learning through Local Trainingcited by 3
  25. 2024FedSheafHN: Personalized Federated Learning on Graph-structured Datacited by 3
  26. 2024Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systemscited by 3
  27. 2025Smoothed Normalization for Efficient Distributed Private Optimizationcited by 3
  28. 2023Privacy Preserving Bayesian Federated Learning in Heterogeneous Settingscited by 2
  29. 2023Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsificationcited by 2
  30. 2024The Power of Bias: Optimizing Client Selection in Federated Learning with Heterogeneous Differential Privacycited by 2
  31. 2025Differentially Private Federated Low Rank Adaptation Beyond Fixed-Matrixcited by 2
  32. 2023Balancing Privacy and Performance for Private Federated Learning Algorithmscited by 1
  33. 2023ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterationscited by 1
  34. 2024On Using Secure Aggregation in Differentially Private Federated Learning with Multiple Local Stepscited by 1
  35. 2025Evaluating Membership Inference Attacks in heterogeneous-data setupscited by 1
  36. 2025Approximate Agreement Algorithms for Byzantine Collaborative Learningcited by 1
  37. 2025Accelerating Differentially Private Federated Learning via Adaptive Extrapolationcited by 1
  38. 2025Federated Instrumental Variable Analysis via Federated Generalized Method of Momentscited by 1

50 most cited of 74 All 74 (JSON)