mlsanityData to 7 Oct 2026

Make Landscape Flatter in Differentially Private Federated Learning

arXiv 2303.11242 ↗First posted 20 Mar 2023cs.LG cs.CR cs.CVcited by 33

Not on the map. Why?

Abstract (arXiv)

To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise. However, existing DPFL methods tend to make a sharper loss landscape and have poorer weight perturbation robustness, resulting in severe performance degradation. To alleviate these issues, we propose a novel DPFL algorithm named DP-FedSAM, which leverages gradient perturbation to mitigate the negative impact of DP. Specifically, DP-FedSAM integrates Sharpness Aware Minimization (SAM) optimizer to generate local flatness models with better stability and weight perturbation robustness, which results in the small norm of local updates and robustness to DP noise, thereby improving the performance. From the theoretical perspective, we analyze in detail how DP-FedSAM mitigates the performance degradation induced by DP. Meanwhile, we give rigorous privacy guarantees with R'enyi DP and present the sensitivity analysis of local updates. At last, we empirically confirm that our algorithm achieves state-of-the-art (SOTA) performance compared with existing SOTA baselines in DPFL. Code is available at https://github.com/YMJS-Irfan/DP-FedSAM

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 43 of its 55 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

43 of 55 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  8. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  9. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  10. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  11. 2020Adaptive Federated Optimizationcited by 1,036
  12. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
31 more
  1. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  2. 2017Renyi Differential Privacycited by 643
  3. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  4. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  5. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  6. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  7. 2019Differentially Private Learning with Adaptive Clippingcited by 218
  8. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  9. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  10. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  11. 2022Towards Understanding Sharpness-Aware Minimizationcited by 145
  12. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  13. 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
  14. 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
  15. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  16. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  17. 2021Decentralized Federated Averagingcited by 114
  18. 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
  19. 2022Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approachcited by 70
  20. 2020LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacycited by 63
  21. 2022Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learningcited by 57
  22. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  23. 2023Improving the Model Consistency of Decentralized Federated Learningcited by 45
  24. 2022Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Modelscited by 42
  25. 2020Federated Model Distillation with Noise-Free Differential Privacycited by 40
  26. 2022Differentially Private Federated Learning with Local Regularization and Sparsificationcited by 35
  27. 2022Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacycited by 32
  28. 2022Achieving Personalized Federated Learning with Sparse Local Modelscited by 30
  29. 2020Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimizationcited by 25
  30. 2023AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networkscited by 21
  31. 2020Voting-based Approaches For Differentially Private Federated Learningcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 11; 2025: 8; 2026: 4.

33 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  3. 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
  4. 2023Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optimacited by 20
  5. 2024A Universal Class of Sharpness-Aware Minimization Algorithmscited by 14
  6. 2024Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challengescited by 14
  7. 2025DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Normcited by 9
  8. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  9. 2024Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learningcited by 8
  10. 2024DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-trainingcited by 7
  11. 2023DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learningcited by 6
  12. 2023Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacycited by 5
21 more
  1. 2023Efficient Federated Learning with Enhanced Privacy via Lottery Ticket Pruning in Edge Computingcited by 5
  2. 2024DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptationcited by 4
  3. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  4. 2023PCDP-SGD: Improving the Convergence of Differentially Private SGD via Projection in Advancecited by 2
  5. 2023Bias-Aware Minimisation: Understanding and Mitigating Estimator Bias in Private SGDcited by 1
  6. 2024FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learningcited by 1
  7. 2025Accelerating Differentially Private Federated Learning via Adaptive Extrapolationcited by 1
  8. 2025Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacycited by 1
  9. 2026DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Modelscited by 1
  10. 2024Convergent Differential Privacy Analysis for General Federated Learningcited by 0
  11. 2024Can We Theoretically Quantify the Impacts of Local Updates on the Generalization Performance of Federated Learning?cited by 0
  12. 2024DP2-FedSAM: Enhancing Differentially Private Federated Learning Through Personalized Sharpness-Aware Minimizationcited by 0
  13. 2024Understanding Generalization of Federated Learning: the Trade-off between Model Stability and Optimizationcited by 0
  14. 2025Differential Privacy Personalized Federated Learning Based on Dynamically Sparsified Client Updatescited by 0
  15. 2025Differentially Private Federated Clustering with Random Rebalancingcited by 0
  16. 2025Federated Learning with Ad-hoc Adapter Insertions: The Case of Soft-Embeddings for Training Classifier-as-Retrievercited by 0
  17. 2025PQFed: A Privacy-Preserving Quality-Controlled Federated Learning Frameworkcited by 0
  18. 2025Optimizer Dynamics at the Edge of Stability with Differential Privacycited by 0
  19. 2026Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregationcited by 0
  20. 2026Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalizationcited by 0
  21. 2026FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learningcited by 0