mlsanityData to 7 Oct 2026

Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data

arXiv 2408.05678 ↗First posted 11 Aug 2024cs.DC cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Despite achieving remarkable performance, Federated Learning (FL) encounters two important problems, i.e., low training efficiency and limited computational resources. In this paper, we propose a new FL framework, i.e., FedDUMAP, with three original contributions, to leverage the shared insensitive data on the server in addition to the distributed data in edge devices so as to efficiently train a global model. First, we propose a simple dynamic server update algorithm, which takes advantage of the shared insensitive data on the server while dynamically adjusting the update steps on the server in order to speed up the convergence and improve the accuracy. Second, we propose an adaptive optimization method with the dynamic server update algorithm to exploit the global momentum on the server and each local device for superior accuracy. Third, we develop a layer-adaptive model pruning method to carry out specific pruning operations, which is adapted to the diverse features of each layer so as to attain an excellent trade-off between effectiveness and efficiency. Our proposed FL model, FedDUMAP, combines the three original techniques and has a significantly better performance compared with baseline approaches in terms of efficiency (up to 16.9 times faster), accuracy (up to 20.4% higher), and computational cost (up to 62.6% smaller).

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 45 of its 82 references that are in the map. First posted on arXiv 11 Aug 2024.

Builds on

45 of 82 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2019Advances and Open Problems in Federated Learningcited by 3,021
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  8. 2018Federated Learning with Non-IID Datacited by 1,099
  9. 2020Adaptive Federated Optimizationcited by 1,036
  10. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  11. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 555
33 more
  1. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 291
  2. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  3. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  4. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
  5. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  6. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  7. 2020Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learningcited by 112
  8. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  9. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  10. 2017On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimizationcited by 91
  11. 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 76
  12. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  13. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
  14. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  15. 2020Federated Composite Optimizationcited by 49
  16. 2019Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Datacited by 38
  17. 2020Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learningcited by 37
  18. 2021Game of Gradients: Mitigating Irrelevant Clients in Federated Learningcited by 35
  19. 2023FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Updatecited by 32
  20. 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
  21. 2021Efficient Device Scheduling with Multi-Job Federated Learningcited by 29
  22. 2022FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Servercited by 29
  23. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  24. 2022FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneitycited by 20
  25. 2022Multi-Job Intelligent Scheduling with Cross-Device Federated Learningcited by 20
  26. 2022FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learningcited by 17
  27. 2021HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environmentscited by 15
  28. 2023AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devicescited by 15
  29. 2023Distributed and Deep Vertical Federated Learning with Big Datacited by 10
  30. 2023Efficient Asynchronous Federated Learning with Sparsification and Quantizationcited by 9
  31. 2024Enhancing Trust and Privacy in Distributed Networks: A Comprehensive Survey on Blockchain-based Federated Learningcited by 8
  32. 2022Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resourcescited by 5
  33. 2021Accelerating Federated Learning with a Global Biased Optimisercited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 1.

6 papers.

  1. 2024Trustworthy Federated Learning: Privacy, Security, and Beyondcited by 11
  2. 2025Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustnesscited by 8
  3. 2024Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelscited by 6
  4. 2025SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignmentcited by 3
  5. 2025Efficient Federated Learning with Heterogeneous Data and Adaptive Dropoutcited by 2
  6. 2026Recovering Clinical Utility Under Differential Privacy: Empirical Validation of Adaptive Federated Aggregation on Heterogeneous Cardiovascular Datasetscited by 0