mlsanityData to 7 Oct 2026

Differentially Private Federated Learning with Local Regularization and Sparsification

arXiv 2203.03106 ↗First posted 7 Mar 2022cs.LG cs.CR cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

User-level differential privacy (DP) provides certifiable privacy guarantees to the information that is specific to any user's data in federated learning. Existing methods that ensure user-level DP come at the cost of severe accuracy decrease. In this paper, we study the cause of model performance degradation in federated learning under user-level DP guarantee. We find the key to solving this issue is to naturally restrict the norm of local updates before executing operations that guarantee DP. To this end, we propose two techniques, Bounded Local Update Regularization and Local Update Sparsification, to increase model quality without sacrificing privacy. We provide theoretical analysis on the convergence of our framework and give rigorous privacy guarantees. Extensive experiments show that our framework significantly improves the privacy-utility trade-off over the state-of-the-arts for federated learning with user-level DP guarantee.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 26 of its 31 references that are in the map. First posted on arXiv 7 Mar 2022.

Builds on

26 of 31 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  8. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  9. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  10. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  11. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  12. 2017Renyi Differential Privacycited by 643
14 more
  1. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  2. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  3. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  4. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  5. 2019Differentially Private Learning with Adaptive Clippingcited by 218
  6. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  7. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  8. 2021Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacycited by 66
  9. 2020LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacycited by 63
  10. 2018Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communicationcited by 51
  11. 2020Federated Model Distillation with Noise-Free Differential Privacycited by 40
  12. 2020Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimizationcited by 25
  13. 2018Variance-based Gradient Compression for Efficient Distributed Deep Learningcited by 21
  14. 2020Voting-based Approaches For Differentially Private Federated Learningcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 12; 2024: 13; 2025: 7; 2026: 2.

35 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2023No One Left Behind: Real-World Federated Class-Incremental Learningcited by 53
  3. 2023Eliminating Domain Bias for Federated Learning in Representation Spacecited by 38
  4. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  5. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 23
  6. 2024Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challengescited by 14
  7. 2023Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learningcited by 13
  8. 2024Prototype Correlation Matching and Class-Relation Reasoning for Few-Shot Medical Image Segmentationcited by 13
  9. 2022Unraveling the Connections between Privacy and Certified Robustness in Federated Learning Against Poisoning Attackscited by 12
  10. 2025DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Normcited by 9
  11. 2024FedUV: Uniformity and Variance for Heterogeneous Federated Learningcited by 8
  12. 2024Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-offcited by 6
23 more
  1. 2023Communication and Energy Efficient Wireless Federated Learning with Intrinsic Privacycited by 5
  2. 2023Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacycited by 5
  3. 2023Efficient Federated Learning with Enhanced Privacy via Lottery Ticket Pruning in Edge Computingcited by 5
  4. 2023Dynamic Privacy Allocation for Locally Differentially Private Federated Learning with Composite Objectivescited by 4
  5. 2024Clip Body and Tail Separately: High Probability Guarantees for DPSGD with Heavy Tailscited by 4
  6. 2024FedCFA: Alleviating Simpson's Paradox in Model Aggregation with Counterfactual Federated Learningcited by 4
  7. 2023Privacy Preservation in Artificial Intelligence and Extended Reality (AI-XR) Metaverses: A Surveycited by 3
  8. 2024Federated Learning for Smart Grid: A Survey on Applications and Potential Vulnerabilitiescited by 3
  9. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  10. 2023PCDP-SGD: Improving the Convergence of Differentially Private SGD via Projection in Advancecited by 2
  11. 2024Privacy-Preserving Federated Learning with Consistency via Knowledge Distillation Using Conditional Generatorcited by 2
  12. 2025Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacycited by 1
  13. 2025Cyst-X: A Multi-Center MRI Benchmark and Federated Learning Framework for Malignancy-Risk Stratification of Pancreatic Cystic Neoplasmcited by 1
  14. 2024DA-PFL: Dynamic Affinity Aggregation for Personalized Federated Learningcited by 0
  15. 2024Federated Hypergraph Learning with Local Differential Privacy: Toward Privacy-Aware Hypergraph Structure Completioncited by 0
  16. 2024Convergent Differential Privacy Analysis for General Federated Learningcited by 0
  17. 2024DP2-FedSAM: Enhancing Differentially Private Federated Learning Through Personalized Sharpness-Aware Minimizationcited by 0
  18. 2025Differential Privacy Personalized Federated Learning Based on Dynamically Sparsified Client Updatescited by 0
  19. 2025UniVarFL: Uniformity and Variance Regularized Federated Learning for Heterogeneous Datacited by 0
  20. 2025Federated Learning with Ad-hoc Adapter Insertions: The Case of Soft-Embeddings for Training Classifier-as-Retrievercited by 0
  21. 2025ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Pushcited by 0
  22. 2026Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregationcited by 0
  23. 2026How Much Regularization Survives Averaging? Update Masking in Federated Learningcited by 0