Corrected with the Latest Version: Make Robust Asynchronous Federated Learning Possible
Not on the map. Why?
Abstract (arXiv)
As an emerging paradigm of federated learning, asynchronous federated learning offers significant speed advantages over traditional synchronous federated learning. Unlike synchronous federated learning, which requires waiting for all clients to complete updates before aggregation, asynchronous federated learning aggregates the models that have arrived in realtime, greatly improving training speed. However, this mechanism also introduces the issue of client model version inconsistency. When the differences between models of different versions during aggregation become too large, it may lead to conflicts, thereby reducing the models accuracy. To address this issue, this paper proposes an asynchronous federated learning version correction algorithm based on knowledge distillation, named FedADT. FedADT applies knowledge distillation before aggregating gradients, using the latest global model to correct outdated information, thus effectively reducing the negative impact of outdated gradients on the training process. Additionally, FedADT introduces an adaptive weighting function that adjusts the knowledge distillation weight according to different stages of training, helps mitigate the misleading effects caused by the poorer performance of the global model in the early stages of training. This method significantly improves the overall performance of asynchronous federated learning without adding excessive computational overhead. We conducted experimental comparisons with several classical algorithms, and the results demonstrate that FedADT achieves significant improvements over other asynchronous methods and outperforms all methods in terms of convergence speed.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 5 Apr 2025.
Builds on
31 of 41 references.
References · 31
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2020Adaptive Federated Optimizationcited by 1,036
- 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
- 2021Model-Contrastive Federated Learningcited by 762
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
- 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
- 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
19 more
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
- 2019Asynchronous Federated Optimizationcited by 264
- 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
- 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
- 2021Federated Learning for Smart Healthcare: A Surveycited by 182
- 2020Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classificationcited by 175
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2019Zero-Shot Knowledge Distillation in Deep Networkscited by 119
- 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
- 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
- 2019SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overheadcited by 82
- 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 76
- 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
- 2019Ensemble Knowledge Distillation for Learning Improved and Efficient Networkscited by 46
- 2022Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillationcited by 45
- 2023FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Updatecited by 32
- 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
- 2023FLGo: A Fully Customizable Federated Learning Platformcited by 12
- 2024FedFa: A Fully Asynchronous Training Paradigm for Federated Learningcited by 12
Built on by
Citing papers per year: 2025: 2.
2 papers.