mlsanityData to 14 Oct 2026

Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation

arXiv 2509.02970 ↗First posted 3 Sep 2025cs.LG math.OCcited by 3

Not on the map. Why?

Abstract (arXiv)

Partial participation is essential for communication-efficient federated learning at scale, yet existing Byzantine-robust methods typically assume full client participation. In the partial participation setting, a majority of the sampled clients may be Byzantine, once Byzantine clients dominate, existing methods break down immediately. We introduce delayed momentum aggregation, a principle where the central server aggregates cached momentum from non-sampled clients along with fresh momentum from sampled clients. This principle ensures Byzantine clients remain a minority from the server's perspective even when they dominate the sampled set. We instantiate this principle in our optimizer DeMoA. We analyze the convergence rate of DeMoA, showing that DeMoA is Byzantine-robust under partial participation. Experiments show that, with 20% Byzantine ratio and only 10% partial participation rate, DeMoA achieves the best accuracy even when existing methods fail empirically.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 77 of its 103 references that are in the map. First posted on arXiv 3 Sep 2025.

Builds on

77 of 103 references.

References · 77

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  6. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 724
  7. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  8. 2018Sparsified SGD with Memorycited by 376
  9. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
  10. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 329
  11. 2019Robust Aggregation for Federated Learningcited by 317
  12. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 288
38 more
  1. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  2. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  3. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  4. 2023Handbook of Convergence Theorems for (Stochastic) Gradient Methodscited by 163
  5. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
  6. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 154
  7. 2017The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learningcited by 150
  8. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 149
  9. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
  10. 2018Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptroncited by 143
  11. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 141
  12. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  13. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 128
  14. 2020Optimal Client Sampling for Federated Learningcited by 124
  15. 2019Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Ratescited by 119
  16. 2020PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimizationcited by 119
  17. 2019Distributed Learning with Compressed Gradient Differencescited by 112
  18. 2018Distributed Stochastic Gradient Tracking Methodscited by 106
  19. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 101
  20. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  21. 2018Byzantine Stochastic Gradient Descentcited by 81
  22. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 79
  23. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  24. 2018DRACO: Byzantine-resilient Distributed Training via Redundant Gradientscited by 75
  25. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  26. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 65
  27. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 64
  28. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  29. 2022Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delayscited by 63
  30. 2020Byzantine-Resilient Non-Convex Stochastic Gradient Descentcited by 57
  31. 2018Distributed learning with compressed gradientscited by 55
  32. 2013Fast Convergence of Stochastic Gradient Descent under a Strong Growth Conditioncited by 54
  33. 2015A Universal Catalyst for First-Order Optimizationcited by 54
  34. 2020Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)cited by 54
  35. 2019Zeno++: Robust Fully Asynchronous SGDcited by 53
  36. 2018On the Ineffectiveness of Variance Reduced Optimization for Deep Learningcited by 48
  37. 2021Acceleration Methodscited by 48
  38. 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Trade-off in Estimating the Number of Byzantine Clients in Federated Learningcited by 0
  2. 2026SWB-DM: A Calibrated Sliced-Wasserstein-Barycenter Aggregator with Delayed-Momentum Caching for Byzantine-Robust Federated Learning under Partial Participationcited by 0
  3. 2026Robustifying Asynchronous SGD via Soft Throttlingcited by 0