# Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data

arXiv: https://arxiv.org/abs/2408.05678 · HTML: https://mlsanity.com/paper/2408.05678/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 82 references that are in the map.
- First posted on arXiv 11 Aug 2024.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [HRank: Filter Pruning using High-Rank Feature Map](https://mlsanity.com/paper/2002.10179/) (2020)
- [Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data](https://mlsanity.com/paper/1811.11479/) (2018)
- [Model Pruning Enables Efficient Federated Learning on Edge Devices](https://mlsanity.com/paper/1909.12326/) (2019)
- [FetchSGD: Communication-Efficient Federated Learning with Sketching](https://mlsanity.com/paper/2007.07682/) (2020)
- [Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning](https://mlsanity.com/paper/2008.03606/) (2020)
- [SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum](https://mlsanity.com/paper/1910.00643/) (2019)
- [Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning](https://mlsanity.com/paper/2006.12097/) (2020)
- [Parameterized Knowledge Transfer for Personalized Federated Learning](https://mlsanity.com/paper/2111.02862/) (2021)
- [FedMix: Approximation of Mixup under Mean Augmented Federated Learning](https://mlsanity.com/paper/2107.00233/) (2021)
- [On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimization](https://mlsanity.com/paper/1708.01012/) (2017)
- [Practical One-Shot Federated Learning for Cross-Silo Setting](https://mlsanity.com/paper/2010.01017/) (2020)
- [Accelerating Federated Learning via Momentum Gradient Descent](https://mlsanity.com/paper/1910.03197/) (2019)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization](https://mlsanity.com/paper/2310.15080/) (2023)
- [Federated Composite Optimization](https://mlsanity.com/paper/2011.08474/) (2020)
- [Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Data](https://mlsanity.com/paper/1905.07210/) (2019)
- [Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning](https://mlsanity.com/paper/2011.04050/) (2020)
- [Game of Gradients: Mitigating Irrelevant Clients in Federated Learning](https://mlsanity.com/paper/2110.12257/) (2021)

## Built on by (most cited first, 6 of 6)

- [Trustworthy Federated Learning: Privacy, Security, and Beyond](https://mlsanity.com/paper/2411.01583/) (2024)
- [Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness](https://mlsanity.com/paper/2510.16171/) (2025)
- [Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models](https://mlsanity.com/paper/2410.00131/) (2024)
- [SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment](https://mlsanity.com/paper/2505.12435/) (2025)
- [Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout](https://mlsanity.com/paper/2507.10430/) (2025)
- [Recovering Clinical Utility Under Differential Privacy: Empirical Validation of Adaptive Federated Aggregation on Heterogeneous Cardiovascular Datasets](https://mlsanity.com/paper/2607.19403/) (2026)

## Abstract (arXiv)

Despite achieving remarkable performance, Federated Learning (FL) encounters two important problems, i.e., low training efficiency and limited computational resources. In this paper, we propose a new FL framework, i.e., FedDUMAP, with three original contributions, to leverage the shared insensitive data on the server in addition to the distributed data in edge devices so as to efficiently train a global model. First, we propose a simple dynamic server update algorithm, which takes advantage of the shared insensitive data on the server while dynamically adjusting the update steps on the server in order to speed up the convergence and improve the accuracy. Second, we propose an adaptive optimization method with the dynamic server update algorithm to exploit the global momentum on the server and each local device for superior accuracy. Third, we develop a layer-adaptive model pruning method to carry out specific pruning operations, which is adapted to the diverse features of each layer so as to attain an excellent trade-off between effectiveness and efficiency. Our proposed FL model, FedDUMAP, combines the three original techniques and has a significantly better performance compared with baseline approaches in terms of efficiency (up to 16.9 times faster), accuracy (up to 20.4% higher), and computational cost (up to 62.6% smaller).