# Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacy

arXiv: https://arxiv.org/abs/2202.07178 · HTML: https://mlsanity.com/paper/2202.07178/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 71 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 30 of 43)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [signSGD: Compressed Optimisation for Non-Convex Problems](https://mlsanity.com/paper/1802.04434/) (2018)
- [Differentially Private Federated Learning: A Client Level Perspective](https://mlsanity.com/paper/1712.07557/) (2017)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Scalable Private Learning with PATE](https://mlsanity.com/paper/1802.08908/) (2018)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Differentially Private Learning with Adaptive Clipping](https://mlsanity.com/paper/1905.03871/) (2019)
- [A Hybrid Approach to Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1812.03224/) (2018)
- [AdaGrad stepsizes: Sharp convergence over nonconvex landscapes](https://mlsanity.com/paper/1806.01811/) (2018)
- [Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity](https://mlsanity.com/paper/1811.12469/) (2018)
- [cpSGD: Communication-efficient and differentially-private distributed SGD](https://mlsanity.com/paper/1805.10559/) (2018)
- [ATOMO: Communication-efficient Learning via Atomic Sparsification](https://mlsanity.com/paper/1806.04090/) (2018)
- [The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation](https://mlsanity.com/paper/2102.06387/) (2021)
- [Tempered Sigmoid Activations for Deep Learning with Differential Privacy](https://mlsanity.com/paper/2007.14191/) (2020)
- [Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification](https://mlsanity.com/paper/2007.03813/) (2020)
- [Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning](https://mlsanity.com/paper/2102.12677/) (2021)
- [Wireless Federated Learning with Local Differential Privacy](https://mlsanity.com/paper/2002.05151/) (2020)
- [DP-ADMM: ADMM-based Distributed Learning with Differential Privacy](https://mlsanity.com/paper/1808.10101/) (2018)
- [Limits of Private Learning with Access to Public Data](https://mlsanity.com/paper/1910.11519/) (2019)
- [FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection](https://mlsanity.com/paper/2003.10637/) (2020)
- [DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation](https://mlsanity.com/paper/2103.11109/) (2021)

## Built on by (most cited first, 30 of 33)

- [Advances and Open Challenges in Federated Foundation Models](https://mlsanity.com/paper/2404.15381/) (2024)
- [Make Landscape Flatter in Differentially Private Federated Learning](https://mlsanity.com/paper/2303.11242/) (2023)
- [Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregation](https://mlsanity.com/paper/2409.01435/) (2024)
- [Detecting Backdoor Attacks in Federated Learning via Direction Alignment Inspection](https://mlsanity.com/paper/2503.07978/) (2025)
- [DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Norm](https://mlsanity.com/paper/2510.27504/) (2025)
- [Federated Progressive Sparsification (Purge, Merge, Tune)+](https://mlsanity.com/paper/2204.12430/) (2022)
- [Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacy](https://mlsanity.com/paper/2309.03437/) (2023)
- [Balancing Privacy Protection and Interpretability in Federated Learning](https://mlsanity.com/paper/2302.08044/) (2023)
- [Communication and Energy Efficient Wireless Federated Learning with Intrinsic Privacy](https://mlsanity.com/paper/2304.07460/) (2023)
- [Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy](https://mlsanity.com/paper/2305.00873/) (2023)
- [Efficient Federated Learning with Enhanced Privacy via Lottery Ticket Pruning in Edge Computing](https://mlsanity.com/paper/2305.01387/) (2023)
- [Differentially Private Federated Learning With Time-Adaptive Privacy Spending](https://mlsanity.com/paper/2502.18706/) (2025)
- [Federated Learning for Smart Grid: A Survey on Applications and Potential Vulnerabilities](https://mlsanity.com/paper/2409.10764/) (2024)
- [Review of Mathematical Optimization in Federated Learning](https://mlsanity.com/paper/2412.01630/) (2024)
- [PRISM: Privacy-Preserving Improved Stochastic Masking for Federated Generative Models](https://mlsanity.com/paper/2503.08085/) (2025)
- [Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification](https://mlsanity.com/paper/2310.19558/) (2023)
- [Heterogeneity-Aware Cooperative Federated Edge Learning with Adaptive Computation and Communication Compression](https://mlsanity.com/paper/2409.04022/) (2024)
- [Private and Communication-Efficient Federated Learning based on Differentially Private Sketches](https://mlsanity.com/paper/2410.05733/) (2024)
- [Federated Hybrid Training and Self-Adversarial Distillation: Towards Robust Edge Networks](https://mlsanity.com/paper/2412.19354/) (2024)
- [Traceable Black-box Watermarks for Federated Learning](https://mlsanity.com/paper/2505.13651/) (2025)
- [Energy-Efficient and Real-Time Sensing for Federated Continual Learning via Sample-Driven Control](https://mlsanity.com/paper/2310.07497/) (2023)
- [Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacy](https://mlsanity.com/paper/2408.04315/) (2024)
- [Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacy](https://mlsanity.com/paper/2505.13655/) (2025)
- [DeepAFL: Deep Analytic Federated Learning](https://mlsanity.com/paper/2603.00579/) (2026)
- [Decentralized Federated Learning by Partial Message Exchange](https://mlsanity.com/paper/2603.01730/) (2026)
- [Convergent Differential Privacy Analysis for General Federated Learning](https://mlsanity.com/paper/2408.15621/) (2024)
- [DP2-FedSAM: Enhancing Differentially Private Federated Learning Through Personalized Sharpness-Aware Minimization](https://mlsanity.com/paper/2409.13645/) (2024)
- [Differential Privacy Personalized Federated Learning Based on Dynamically Sparsified Client Updates](https://mlsanity.com/paper/2503.09192/) (2025)
- [Moss: Proxy Model-based Full-Weight Aggregation in Federated Learning with Heterogeneous Models](https://mlsanity.com/paper/2503.10218/) (2025)
- [Robust Federated Learning against Model Perturbation in Edge Networks](https://mlsanity.com/paper/2505.24728/) (2025)

## Abstract (arXiv)

Federated learning (FL) that enables edge devices to collaboratively learn a shared model while keeping their training data locally has received great attention recently and can protect privacy in comparison with the traditional centralized learning paradigm. However, sensitive information about the training data can still be inferred from model parameters shared in FL. Differential privacy (DP) is the state-of-the-art technique to defend against those attacks. The key challenge to achieving DP in FL lies in the adverse impact of DP noise on model accuracy, particularly for deep learning models with large numbers of parameters. This paper develops a novel differentially-private FL scheme named Fed-SMP that provides a client-level DP guarantee while maintaining high model accuracy. To mitigate the impact of privacy protection on model accuracy, Fed-SMP leverages a new technique called Sparsified Model Perturbation (SMP) where local models are sparsified first before being perturbed by Gaussian noise. We provide a tight end-to-end privacy analysis for Fed-SMP using Renyi DP and prove the convergence of Fed-SMP with both unbiased and biased sparsifications. Extensive experiments on real-world datasets are conducted to demonstrate the effectiveness of Fed-SMP in improving model accuracy with the same DP guarantee and saving communication cost simultaneously.