mlsanityData to 14 Oct 2026

Aggregating Capacity in FL through Successive Layer Training for Computationally-Constrained Devices

arXiv 2305.17005 ↗First posted 26 May 2023cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is usually performed on resource-constrained edge devices, e.g., with limited memory for the computation. If the required memory to train a model exceeds this limit, the device will be excluded from the training. This can lead to a lower accuracy as valuable data and computation resources are excluded from training, also causing bias and unfairness. The FL training process should be adjusted to such constraints. The state-of-the-art techniques propose training subsets of the FL model at constrained devices, reducing their resource requirements for training. But these techniques largely limit the co-adaptation among parameters of the model and are highly inefficient, as we show: it is actually better to train a smaller (less accurate) model by the system where all the devices can train the model end-to-end, than applying such techniques. We propose a new method that enables successive freezing and training of the parameters of the FL model at devices, reducing the training's resource requirements at the devices, while still allowing enough co-adaptation between parameters. We show through extensive experimental evaluation that our technique greatly improves the accuracy of the trained model (by 52.4 p.p.) compared with the state of the art, efficiently aggregating the computation capacity available on distributed devices.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 26 May 2023.

Builds on

30 of 44 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  6. 2018LEAF: A Benchmark for Federated Settingscited by 792
  7. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  8. 2019Asynchronous Federated Optimizationcited by 267
  9. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 204
  10. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  11. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  12. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
18 more
  1. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
  2. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  3. 2020TiFL: A Tier-based Federated Learning Systemcited by 96
  4. 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 86
  5. 2019Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systemscited by 78
  6. 2020LoCo: Local Contrastive Representation Learningcited by 55
  7. 2020Dynamic Tensor Rematerializationcited by 46
  8. 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
  9. 2018Accelerating Convolutional Neural Networks via Activation Map Compressioncited by 34
  10. 2021FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorizationcited by 31
  11. 2021ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Trainingcited by 27
  12. 2020A Federated Learning Framework for Healthcare IoT devicescited by 22
  13. 2022Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneitycited by 17
  14. 2017Forward Thinking: Building and Training Neural Networks One Layer at a Timecited by 15
  15. 2021DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systemscited by 10
  16. 2020Federated Learning for Localization: A Privacy-Preserving Crowdsourcing Methodcited by 9
  17. 2019Federated Imitation Learning: A Privacy Considered Imitation Learning Framework for Cloud Robotic Systems with Heterogeneous Sensor Datacited by 4
  18. 2022FedNet2Net: Saving Communication and Computations in Federated Learning with Model Growingcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 4.

7 papers.

  1. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  2. 2024FedSPU: Personalized Federated Learning for Resource-constrained Devices with Stochastic Parameter Updatecited by 3
  3. 2024Efficient Federated Finetuning of Tiny Transformers with Resource-Constrained Devicescited by 1
  4. 2025Accelerated Training on Low-Power Edge Devicescited by 1
  5. 2025Bridging Generalization Gap of Heterogeneous Federated Clients Using Generative Modelscited by 1
  6. 2025Energy and Memory-Efficient Federated Learning With Ordered Layer Freezingcited by 1
  7. 2025Efficient Zeroth-Order Federated Finetuning of Language Models on Resource-Constrained Devicescited by 0