mlsanityData to 14 Oct 2026

FedOptima: Optimizing Resource Utilization in Federated Learning

arXiv 2504.13850 ↗First posted 10 Mar 2025cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) systems facilitate distributed machine learning across a server and multiple devices. However, FL systems have low resource utilization on servers and devices, limiting their practical use in the real world. This inefficiency primarily arises from two types of idle time: (i) task dependency between the server and devices, and (ii) stragglers among heterogeneous devices. This paper introduces FedOptima, a resource-optimized FL system designed to simultaneously minimize both types of idle time; existing systems do not eliminate or reduce both at the same time. FedOptima offloads the training of certain layers of a neural network from a device to a server using three innovations. First, devices operate independently of each other using asynchronous aggregation to eliminate straggler effects, and independently of the server by utilizing auxiliary networks to minimize idle time caused by task dependency. Second, the server performs centralized training using a task scheduler that ensures balanced contributions from all devices, improving model accuracy. Third, an efficient memory management mechanism on the server increases the scalability of the number of participating devices. Extensive experiments are conducted on multiple lab-based testbeds, evaluated on image classification and sentiment analysis tasks with CNNs and Transformers. Compared to four state-of-the-art offloading-based and asynchronous FL baselines, FedOptima (i) achieves higher or comparable accuracy, (ii) accelerates training by 1.9x to 21.8x, (iii) reduces server and device idle time by up to 93.9% and 81.8%, respectively, and (iv) increases throughput by 1.1x to 2.0x.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 39 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

29 of 39 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  6. 2016Deep Learning with Differential Privacycited by 2,728
  7. 2019Searching for MobileNetV3cited by 1,994
  8. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  9. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 564
  10. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  11. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 468
  12. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 395
17 more
  1. 2020SplitFed: When Federated Learning Meets Split Learningcited by 310
  2. 2018Distributed learning of deep neural network over multiple agentscited by 279
  3. 2019Asynchronous Federated Optimizationcited by 267
  4. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
  5. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 195
  6. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 190
  7. 2019Decoupled Greedy Learning of CNNscited by 78
  8. 2022Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customizationcited by 35
  9. 2021FedAdapt: Adaptive Offloading for IoT Devices in Federated Learningcited by 26
  10. 2021Resource-Efficient Federated Learningcited by 26
  11. 2018Training Neural Networks Using Features Replaycited by 23
  12. 2021Federated Learning for Internet of Things: A Federated Learning Framework for On-device Anomaly Data Detectioncited by 20
  13. 2022SplitGP: Achieving Both Generalization and Personalization in Federated Learningcited by 20
  14. 2022Robust Split Federated Learning for U-shaped Medical Image Networkscited by 11
  15. 2023Communication and Storage Efficient Federated Split Learningcited by 11
  16. 2024FedStaleWeight: Buffered Asynchronous Federated Learning with Fair Aggregation via Staleness Reweightingcited by 5
  17. 2022PiPar: Pipeline Parallelism for Collaborative Machine Learningcited by 2

Built on by

Not cited yet.