mlsanityData to 7 Oct 2026

Evaluation and Optimization of Distributed Machine Learning Techniques for Internet of Things

arXiv 2103.02762 ↗First posted 3 Mar 2021cs.LG cs.CR cs.NIcited by 20

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) and split learning (SL) are state-of-the-art distributed machine learning techniques to enable machine learning training without accessing raw data on clients or end devices. However, their comparative training performance under real-world resource-restricted Internet of Things (IoT) device settings, e.g., Raspberry Pi, remains barely studied, which, to our knowledge, have not yet been evaluated and compared, rendering inconvenient reference for practitioners. This work firstly provides empirical comparisons of FL and SL in real-world IoT settings regarding (i) learning performance with heterogeneous data distributions and (ii) on-device execution overhead. Our analyses in this work demonstrate that the learning performance of SL is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. Recently, FL and SL are combined to form splitfed learning (SFL) to leverage each of their benefits (e.g., parallel training of FL and lightweight on-device computation requirement of SL). This work then considers FL, SL, and SFL, and mount them on Raspberry Pi devices to evaluate their performance, including training time, communication overhead, power consumption, and memory usage. Besides evaluations, we apply two optimizations. Firstly, we generalize SFL by carefully examining the possibility of a hybrid type of model training at the server-side. The generalized SFL merges sequential (dependent) and parallel (independent) processes of model training and is thus beneficial for a system with large-scaled IoT devices, specifically at the server-side operations. Secondly, we propose pragmatic techniques to substantially reduce the communication overhead by up to four times for the SL and (generalized) SFL.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 20 of its 52 references that are in the map. First posted on arXiv 3 Mar 2021.

Builds on

20 of 52 references.

References · 20

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  8. 2019Towards Federated Learning at Scale: System Designcited by 826
  9. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
  10. 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
  11. 2018Distributed learning of deep neural network over multiple agentscited by 277
  12. 20191D Convolutional Neural Networks and Applications: A Surveycited by 152
8 more
  1. 2018D"IoT: A Federated Self-learning Anomaly Detection System for IoTcited by 89
  2. 2019SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overheadcited by 82
  3. 2019Detailed comparison of communication efficiency of split learning and federated learningcited by 76
  4. 2019Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Recordscited by 70
  5. 2020End-to-End Evaluation of Federated Learning and Split Learning for Internet of Thingscited by 64
  6. 2020Advancements of federated learning towards privacy preservation: from federated learning to split learningcited by 34
  7. 2018No Peek: A Survey of private distributed deep learningcited by 30
  8. 2016Adversary Resistant Deep Neural Networks with an Application to Malware Detectioncited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 5; 2024: 5; 2025: 4; 2026: 1.

20 papers.

Sort:

By area:

  1. 2022Combined Federated and Split Learning in Edge Computing for Ubiquitous Intelligence in Internet of Things: State of the Art and Future Directionscited by 14
  2. 2022Differentially Private CutMix for Split Learning with Vision Transformercited by 13
  3. 2022Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learningcited by 11
  4. 2022Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learningcited by 8
  5. 2023Vertical Federated Learning: Taxonomies, Threats, and Prospectscited by 8
  6. 2024Model Partition and Resource Allocation for Split Learning in Vehicular Edge Networkscited by 6
  7. 2024COMSPLIT: A Communication-Aware Split Learning Design for Heterogeneous IoT Platformscited by 5
  8. 2023When MiniBatch SGD Meets SplitFed Learning:Convergence Analysis and Performance Evaluationcited by 3
  9. 2024Parallel Split Learning with Global Samplingcited by 3
  10. 2024CRSFL: Cluster-based Resource-aware Split Federated Learning for Continuous Authenticationcited by 2
  11. 2024Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMixcited by 2
  12. 2025Data Heterogeneity and Forgotten Labels in Split Federated Learningcited by 2
8 more
  1. 2025A Novel Collaborative Framework for Efficient Synchronization in Split Federated Learning over Wireless Networkscited by 1
  2. 2025RockNet: Distributed Learning on Ultra-Low-Power Devicescited by 1
  3. 2022Distributed Machine Learning in D2D-Enabled Heterogeneous Networks: Architectures, Performance, and Open Challengescited by 0
  4. 2023When Computing Power Network Meets Distributed Machine Learning: An Efficient Federated Split Learning Frameworkcited by 0
  5. 2023Federated Split Learning with Only Positive Labels for resource-constrained IoT environmentcited by 0
  6. 2023Enabling Intelligent Vehicular Networks Through Distributed Learning in the Non-Terrestrial Networks 6G Visioncited by 0
  7. 2025Modular Distributed Nonconvex Learning with Error Feedbackcited by 0
  8. 2026Makespan Minimization in Split Learning: From Theory to Practicecited by 0