mlsanityData to 7 Oct 2026

Stragglers Can Contribute More: Uncertainty-Aware Distillation for Asynchronous Federated Learning

arXiv 2511.19966 ↗First posted 25 Nov 2025cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Asynchronous federated learning (FL) has recently gained attention for its enhanced efficiency and scalability, enabling local clients to send model updates to the server at their own pace without waiting for slower participants. However, such a design encounters significant challenges, such as the risk of outdated updates from straggler clients degrading the overall model performance and the potential bias introduced by faster clients dominating the learning process, especially under heterogeneous data distributions. Existing methods typically address only one of these issues, creating a conflict where mitigating the impact of outdated updates can exacerbate the bias created by faster clients, and vice versa. To address these challenges, we propose FedEcho, a novel framework that incorporates uncertainty-aware distillation to enhance the asynchronous FL performances under large asynchronous delays and data heterogeneity. Specifically, uncertainty-aware distillation enables the server to assess the reliability of predictions made by straggler clients, dynamically adjusting the influence of these predictions based on their estimated uncertainty. By prioritizing more certain predictions while still leveraging the diverse information from all clients, FedEcho effectively mitigates the negative impacts of outdated updates and data heterogeneity. Through extensive experiments, we demonstrate that FedEcho consistently outperforms existing asynchronous federated learning baselines, achieving robust performance without requiring access to private client data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 48 references that are in the map. First posted on arXiv 25 Nov 2025.

Builds on

36 of 48 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  7. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  8. 2020Federated Learning with Matched Averagingcited by 564
  9. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  10. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  11. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  12. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
24 more
  1. 2019Asynchronous Federated Optimizationcited by 264
  2. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  3. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  4. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  5. 2018On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimizationcited by 173
  6. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  7. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  8. 2018SGD and Hogwild! Convergence Without the Bounded Gradients Assumptioncited by 94
  9. 2023One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillationcited by 77
  10. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 76
  11. 2015Perturbed Iterate Analysis for Asynchronous Stochastic Optimizationcited by 67
  12. 2022Communication-Efficient Adaptive Federated Learningcited by 55
  13. 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
  14. 2021Anarchic Federated Learningcited by 42
  15. 2023FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Updatecited by 32
  16. 2018Improved asynchronous parallel optimization analysis for stochastic incremental methodscited by 30
  17. 2021Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updatescited by 16
  18. 2022Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregationcited by 11
  19. 2024Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learningcited by 11
  20. 2023Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregationcited by 10
  21. 2024FADAS: Towards Federated Adaptive Asynchronous Optimizationcited by 10
  22. 2024Momentum Approximation in Asynchronous Private Federated Learningcited by 3
  23. 2024HYDRA-FL: Hybrid Knowledge Distillation for Robust and Accurate Federated Learningcited by 2
  24. 2024Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Datacited by 1

Built on by

Not cited yet.