mlsanityData to 7 Oct 2026

Adaptive Gradient Clipping for Robust Federated Learning

arXiv 2405.14432 ↗First posted 23 May 2024cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Robust federated learning aims to maintain reliable performance despite the presence of adversarial or misbehaving workers. While state-of-the-art (SOTA) robust distributed gradient descent (Robust-DGD) methods were proven theoretically optimal, their empirical success has often relied on pre-aggregation gradient clipping. However, existing static clipping strategies yield inconsistent results: enhancing robustness against some attacks while being ineffective or even detrimental against others. To address this limitation, we propose a principled adaptive clipping strategy, Adaptive Robust Clipping (ARC), which dynamically adjusts clipping thresholds based on the input gradients. We prove that ARC not only preserves the theoretical robustness guarantees of SOTA Robust-DGD methods but also provably improves asymptotic convergence when the model is well-initialized. Extensive experiments on benchmark image classification tasks confirm these theoretical insights, demonstrating that ARC significantly enhances robustness, particularly in highly heterogeneous and adversarial settings.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 37 of its 53 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

37 of 53 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2021Convex optimizationcited by 2,479
  7. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  9. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  10. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
  11. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 285
  12. 2021Federated Learning for Smart Healthcare: A Surveycited by 182
25 more
  1. 2018Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptroncited by 143
  2. 2021Federated Learning for Open Bankingcited by 138
  3. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  4. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 136
  5. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 125
  6. 2020Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clippingcited by 104
  7. 2020Improved Analysis of Clipping Algorithms for Non-convex Optimizationcited by 88
  8. 2023Revisiting Gradient Clipping: Stochastic bias and tight convergence guaranteescited by 83
  9. 2019AdaCliP: Adaptive Clipping for Private SGDcited by 77
  10. 2021Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacycited by 66
  11. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 64
  12. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  13. 2020Byzantine-Resilient Non-Convex Stochastic Gradient Descentcited by 56
  14. 2020Outlier Robust Mean Estimation with Subgaussian Rates via Stabilitycited by 50
  15. 2021Approximate Byzantine Fault-Tolerance in Distributed Optimizationcited by 27
  16. 2022On the Impossible Safety of Large AI Modelscited by 27
  17. 2023Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneitycited by 25
  18. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  19. 2021Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?cited by 17
  20. 2022Covariance Estimation: Optimal Dimension-free Guarantees for Adversarial Corruption and Heavy Tailscited by 14
  21. 2022Clipped Stochastic Methods for Variational Inequalities with Heavy-Tailed Noisecited by 13
  22. 2022Bridging Differential Privacy and Byzantine-Robustness via Model Aggregationcited by 12
  23. 2022Robust Collaborative Learning with Linear Gradient Overheadcited by 12
  24. 2023Clip21: Error Feedback for Gradient Clippingcited by 10
  25. 2023Balancing Privacy, Robustness, and Efficiency in Machine Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 7.

10 papers.

  1. 2026TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizerscited by 6
  2. 2025ByzFL: Research Framework for Robust Federated Learningcited by 3
  3. 2025Tight Stability Bounds for Robust Distributed Learning: Byzantine Failures Hurt Generalization More than Data Poisoningcited by 2
  4. 2025ProDiGy: Proximity- and Dissimilarity-Based Byzantine-Robust Federated Learningcited by 0
  5. 2026TinyGuard:A lightweight Byzantine Defense for Resource-Constrained Federated Learning via Statistical Update Fingerprintscited by 0
  6. 2026Contract And Conquer: How to Provably Compute Adversarial Examples for a Black-Box Model?cited by 0
  7. 2026Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Boundscited by 0
  8. 2026Practical Validity Conditions for Byzantine-Tolerant Federated Learningcited by 0
  9. 2026Dangerous Liaisons of Convex Learning and Non-Affine Aggregationcited by 0
  10. 2026On the Gradient Heterogeneity Dynamics of Adversarially Robust Federated Regressioncited by 0