mlsanityData to 7 Oct 2026

Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacy

arXiv 2202.07178 ↗First posted 15 Feb 2022cs.LG cs.CRcited by 32

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) that enables edge devices to collaboratively learn a shared model while keeping their training data locally has received great attention recently and can protect privacy in comparison with the traditional centralized learning paradigm. However, sensitive information about the training data can still be inferred from model parameters shared in FL. Differential privacy (DP) is the state-of-the-art technique to defend against those attacks. The key challenge to achieving DP in FL lies in the adverse impact of DP noise on model accuracy, particularly for deep learning models with large numbers of parameters. This paper develops a novel differentially-private FL scheme named Fed-SMP that provides a client-level DP guarantee while maintaining high model accuracy. To mitigate the impact of privacy protection on model accuracy, Fed-SMP leverages a new technique called Sparsified Model Perturbation (SMP) where local models are sparsified first before being perturbed by Gaussian noise. We provide a tight end-to-end privacy analysis for Fed-SMP using Renyi DP and prove the convergence of Fed-SMP with both unbiased and biased sparsifications. Extensive experiments on real-world datasets are conducted to demonstrate the effectiveness of Fed-SMP in improving model accuracy with the same DP guarantee and saving communication cost simultaneously.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 43 of its 71 references that are in the map. First posted on arXiv 15 Feb 2022.

Builds on

43 of 71 references.

References · 43

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  8. 2020Adaptive Federated Optimizationcited by 1,036
  9. 2019Towards Federated Learning at Scale: System Designcited by 826
  10. 2017Renyi Differential Privacycited by 643
  11. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  12. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
31 more
  1. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  2. 2018Scalable Private Learning with PATEcited by 266
  3. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
  4. 2019Differentially Private Learning with Adaptive Clippingcited by 218
  5. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  6. 2018AdaGrad stepsizes: Sharp convergence over nonconvex landscapescited by 182
  7. 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 163
  8. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  9. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 135
  10. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  11. 2020Tempered Sigmoid Activations for Deep Learning with Differential Privacycited by 121
  12. 2020Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identificationcited by 88
  13. 2021Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learningcited by 86
  14. 2020Wireless Federated Learning with Local Differential Privacycited by 50
  15. 2018DP-ADMM: ADMM-based Distributed Learning with Differential Privacycited by 41
  16. 2019Limits of Private Learning with Access to Public Datacited by 38
  17. 2020FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selectioncited by 35
  18. 2021DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregationcited by 31
  19. 2020Recent Theoretical Advances in Non-Convex Optimizationcited by 30
  20. 2020FLAME: Differentially Private Federated Learning in the Shuffle Modelcited by 27
  21. 2020Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimizationcited by 25
  22. 2019Federated Learning with Autotuned Communication-Efficient Secure Aggregationcited by 23
  23. 2021ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Trainingcited by 22
  24. 2020Two-Phase Multi-Party Computation Enabled Privacy-Preserving Federated Learningcited by 19
  25. 2020Private and Communication-Efficient Edge Learning: A Sparse Differential Gaussian-Masking Distributed SGD Approachcited by 18
  26. 2019Privacy for Free: Communication-Efficient Learning with Differential Privacy Using Sketchescited by 17
  27. 2020Compression Boosts Differentially Private Federated Learningcited by 13
  28. 2021DP-SIGNSGD: When Efficiency Meets Privacy and Robustnesscited by 11
  29. 2020Differentially Private Federated Learning with Laplacian Smoothingcited by 8
  30. 2017Computational Differential Privacy from Lattice-based Cryptographycited by 6
  31. 2020Efficient Sparse Secure Aggregation for Federated Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 10; 2025: 9; 2026: 4.

32 papers.

Sort:

  1. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  2. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  3. 2024Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregationcited by 13
  4. 2025Detecting Backdoor Attacks in Federated Learning via Direction Alignment Inspectioncited by 13
  5. 2025DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Normcited by 9
  6. 2022Federated Progressive Sparsification (Purge, Merge, Tune)+cited by 8
  7. 2023Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacycited by 8
  8. 2023Balancing Privacy Protection and Interpretability in Federated Learningcited by 6
  9. 2023Communication and Energy Efficient Wireless Federated Learning with Intrinsic Privacycited by 5
  10. 2023Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacycited by 5
  11. 2023Efficient Federated Learning with Enhanced Privacy via Lottery Ticket Pruning in Edge Computingcited by 5
  12. 2025Differentially Private Federated Learning With Time-Adaptive Privacy Spendingcited by 5
20 more
  1. 2024Federated Learning for Smart Grid: A Survey on Applications and Potential Vulnerabilitiescited by 3
  2. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  3. 2025PRISM: Privacy-Preserving Improved Stochastic Masking for Federated Generative Modelscited by 3
  4. 2023Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsificationcited by 2
  5. 2024Heterogeneity-Aware Cooperative Federated Edge Learning with Adaptive Computation and Communication Compressioncited by 2
  6. 2024Federated Hybrid Training and Self-Adversarial Distillation: Towards Robust Edge Networkscited by 2
  7. 2023Energy-Efficient and Real-Time Sensing for Federated Continual Learning via Sample-Driven Controlcited by 1
  8. 2024Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacycited by 1
  9. 2024Private and Communication-Efficient Federated Learning based on Differentially Private Sketchescited by 1
  10. 2025Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacycited by 1
  11. 2026DeepAFL: Deep Analytic Federated Learningcited by 1
  12. 2026Decentralized Federated Learning by Partial Message Exchangecited by 1
  13. 2024Convergent Differential Privacy Analysis for General Federated Learningcited by 0
  14. 2024DP2-FedSAM: Enhancing Differentially Private Federated Learning Through Personalized Sharpness-Aware Minimizationcited by 0
  15. 2025Differential Privacy Personalized Federated Learning Based on Dynamically Sparsified Client Updatescited by 0
  16. 2025Moss: Proxy Model-based Full-Weight Aggregation in Federated Learning with Heterogeneous Modelscited by 0
  17. 2025Traceable Black-box Watermarks for Federated Learningcited by 0
  18. 2025Robust Federated Learning against Model Perturbation in Edge Networkscited by 0
  19. 2026An Adaptive Differentially Private Federated Learning Frameworkcited by 0
  20. 2026Key-Embedded Privacy for Decentralized AI in Biomedical Omicscited by 0