mlsanityData to 14 Oct 2026

Knowledge Rumination for Client Utility Evaluation in Heterogeneous Federated Learning

arXiv 2312.10425 ↗First posted 16 Dec 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) allows several clients to cooperatively train machine learning models without disclosing the raw data. In practical applications, asynchronous FL (AFL) can address the straggler effect compared to synchronous FL. However, Non-IID data and stale models pose significant challenges to AFL, as they can diminish the practicality of the global model and even lead to training failures. In this work, we propose a novel AFL framework called Federated Historical Learning (FedHist), which effectively addresses the challenges posed by both Non-IID data and gradient staleness based on the concept of knowledge rumination. FedHist enhances the stability of local gradients by performing weighted fusion with historical global gradients cached on the server. Relying on hindsight, it assigns aggregation weights to each participant in a multi-dimensional manner during each communication round. To further enhance the efficiency and stability of the training process, we introduce an intelligent ℓ2-norm amplification scheme, which dynamically regulates the learning progress based on the ℓ2-norms of the submitted gradients. Extensive experiments indicate FedHist outperforms state-of-the-art methods in terms of convergence performance and test accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 16 Dec 2023.

Builds on

20 of 27 references.

References · 20

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2018Federated Learning with Non-IID Datacited by 1,108
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  6. 2021Model-Contrastive Federated Learningcited by 766
  7. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  8. 2019Asynchronous Federated Optimizationcited by 267
  9. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 100
  10. 2019SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overheadcited by 82
  11. 2019Device Scheduling with Fast Convergence for Wireless Federated Learningcited by 46
  12. 2022FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clientscited by 46
8 more
  1. 2019Federated Transfer Reinforcement Learning for Autonomous Drivingcited by 44
  2. 2023TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Trainingcited by 27
  3. 2021Resource-Efficient Federated Learningcited by 26
  4. 2022FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregationcited by 21
  5. 2022FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneitycited by 20
  6. 2021Coordinating Momenta for Cross-silo Federated Learningcited by 9
  7. 2022Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Datacited by 9
  8. 2019Gradient Scheduling with Global Momentum for Non-IID Data Distributed Asynchronous Trainingcited by 5

Built on by

Not cited yet.