mlsanityData to 7 Oct 2026

A Resource-Adaptive Approach for Federated Learning under Resource-Constrained Environments

arXiv 2406.13351 ↗First posted 19 Jun 2024cs.LG cs.AI cs.DCcited by 1

Not on the map. Why?

Abstract (arXiv)

The paper studies a fundamental federated learning (FL) problem involving multiple clients with heterogeneous constrained resources. Compared with the numerous training parameters, the computing and communication resources of clients are insufficient for fast local training and real-time knowledge sharing. Besides, training on clients with heterogeneous resources may result in the straggler problem. To address these issues, we propose Fed-RAA: a Resource-Adaptive Asynchronous Federated learning algorithm. Different from vanilla FL methods, where all parameters are trained by each participating client regardless of resource diversity, Fed-RAA adaptively allocates fragments of the global model to clients based on their computing and communication capabilities. Each client then individually trains its assigned model fragment and asynchronously uploads the updated result. Theoretical analysis confirms the convergence of our approach. Additionally, we design an online greedy-based algorithm for fragment allocation in Fed-RAA, achieving fairness comparable to an offline strategy. We present numerical results on MNIST, CIFAR-10, and CIFAR-100, along with necessary comparisons and ablation studies, demonstrating the advantages of our work. To the best of our knowledge, this paper represents the first resource-adaptive asynchronous method for fragment-based FL with guaranteed theoretical convergence.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 19 Jun 2024.

Builds on

18 of 27 references.

References · 18

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  5. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  6. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  7. 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
  8. 2019Asynchronous Federated Optimizationcited by 264
  9. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  10. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  11. 2019Distributed Learning of Deep Neural Networks using Independent Subnet Trainingcited by 20
  12. 2024Modeling Other Players with Bayesian Beliefs for Games with Incomplete Informationcited by 16
6 more
  1. 2024Resource Allocation in Large Language Model Integrated 6G Vehicular Networkscited by 11
  2. 2022Online Model Compression for Federated Learning with Large Modelscited by 7
  3. 2024Collaborative AI Teaming in Unknown Environments via Active Goal Deductioncited by 6
  4. 2024Cooperative Backdoor Attack in Decentralized Reinforcement Learning with Theoretical Guaranteecited by 6
  5. 2022Split Two-Tower Model for Efficient and Privacy-Preserving Cross-device Federated Recommendationcited by 5
  6. 2022FedDCT: Federated Learning of Large Convolutional Neural Networks on Resource Constrained Devices using Divide and Collaborative Trainingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026RefProtoFL: Communication-Efficient Federated Learning via External-Referenced Prototype Alignmentcited by 0