mlsanityData to 14 Oct 2026

DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Norm

arXiv 2510.27504 ↗First posted 31 Oct 2025cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

To prevent inference attacks in Federated Learning (FL) and reduce the leakage of sensitive information, Client-level Differentially Private Federated Learning (CL-DPFL) is widely used. However, current CL-DPFL methods usually result in sharper loss landscapes, which leads to a decrease in model generalization after differential privacy protection. By using Sharpness Aware Minimization (SAM), the current popular federated learning methods are to find a local flat minimum value to alleviate this problem. However, the local flatness may not reflect the global flatness in CL-DPFL. Therefore, to address this issue and seek global flat minima of models, we propose a new CL-DPFL algorithm, DP-FedPGN, in which we introduce a global gradient norm penalty to the local loss to find the global flat minimum. Moreover, by using our global gradient norm penalty, we not only find a flatter global minimum but also reduce the locally updated norm, which means that we further reduce the error of gradient clipping. From a theoretical perspective, we analyze how DP-FedPGN mitigates the performance degradation caused by DP. Meanwhile, the proposed DP-FedPGN algorithm eliminates the impact of data heterogeneity and achieves fast convergence. We also use R'enyi DP to provide strict privacy guarantees and provide sensitivity analysis for local updates. Finally, we conduct effectiveness tests on both ResNet and Transformer models, and achieve significant improvements in six visual and natural language processing tasks compared to existing state-of-the-art algorithms. The code is available at https://github.com/junkangLiu0/DP-FedPGN

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 31 Oct 2025.

Builds on

27 of 44 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  7. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  8. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  9. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  10. 2017Renyi Differential Privacycited by 650
  11. 2017Exploring Generalization in Deep Learningcited by 600
  12. 2017Sharp Minima Can Generalize For Deep Netscited by 466
15 more
  1. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  2. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 415
  3. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  4. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 128
  5. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  6. 2021Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacycited by 69
  7. 2022Differentially Private Federated Learning with Local Regularization and Sparsificationcited by 36
  8. 2022Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacycited by 33
  9. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  10. 2025FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averagingcited by 25
  11. 2024Prompt Categories Cluster for Weakly Supervised Semantic Segmentationcited by 16
  12. 2019DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERMcited by 15
  13. 2020Tighter Generalization Bounds for Iterative Differentially Private Learning Algorithmscited by 14
  14. 2025LLM-Enhanced Multimodal Fusion for Cross-Domain Sequential Recommendationcited by 11
  15. 2020Differentially Private Federated Learning with Laplacian Smoothingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 8.

9 papers.

  1. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  2. 2026FedNSAM:Consistency of Local and Global Flatness for Federated Learningcited by 7
  3. 2026Rethinking LoRA for Privacy-Preserving Federated Learning in Large Modelscited by 5
  4. 2026DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Modelscited by 1
  5. 2026Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosiscited by 1
  6. 2026SeFoRA: Sketch-Aggregated Federated Low-Rank Adaptation with Heterogeneous Client Rankscited by 1
  7. 2026CTP-FL: Common-Trajectory Gradient Prediction for Federated Learningcited by 0
  8. 2026FedMIX-P: Mixing Local and Global Preconditioners for Federated Vision and Language Model Trainingcited by 0
  9. 2026FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projectioncited by 0