# Evaluation and Optimization of Distributed Machine Learning Techniques for Internet of Things

arXiv: https://arxiv.org/abs/2103.02762 · HTML: https://mlsanity.com/paper/2103.02762/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 20 of 20)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [1D Convolutional Neural Networks and Applications: A Survey](https://mlsanity.com/paper/1905.03554/) (2019)
- [D"IoT: A Federated Self-learning Anomaly Detection System for IoT](https://mlsanity.com/paper/1804.07474/) (2018)
- [SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overhead](https://mlsanity.com/paper/1910.01355/) (2019)
- [Detailed comparison of communication efficiency of split learning and federated learning](https://mlsanity.com/paper/1909.09145/) (2019)
- [Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Records](https://mlsanity.com/paper/1903.09296/) (2019)
- [End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things](https://mlsanity.com/paper/2003.13376/) (2020)
- [Advancements of federated learning towards privacy preservation: from federated learning to split learning](https://mlsanity.com/paper/2011.14818/) (2020)
- [No Peek: A Survey of private distributed deep learning](https://mlsanity.com/paper/1812.03288/) (2018)
- [Adversary Resistant Deep Neural Networks with an Application to Malware Detection](https://mlsanity.com/paper/1610.01239/) (2016)

## Built on by (most cited first, 20 of 20)

- [Combined Federated and Split Learning in Edge Computing for Ubiquitous Intelligence in Internet of Things: State of the Art and Future Directions](https://mlsanity.com/paper/2207.09611/) (2022)
- [Differentially Private CutMix for Split Learning with Vision Transformer](https://mlsanity.com/paper/2210.15986/) (2022)
- [Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learning](https://mlsanity.com/paper/2209.02428/) (2022)
- [Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learning](https://mlsanity.com/paper/2207.00234/) (2022)
- [Vertical Federated Learning: Taxonomies, Threats, and Prospects](https://mlsanity.com/paper/2302.01550/) (2023)
- [Model Partition and Resource Allocation for Split Learning in Vehicular Edge Networks](https://mlsanity.com/paper/2411.06773/) (2024)
- [COMSPLIT: A Communication-Aware Split Learning Design for Heterogeneous IoT Platforms](https://mlsanity.com/paper/2410.19375/) (2024)
- [When MiniBatch SGD Meets SplitFed Learning:Convergence Analysis and Performance Evaluation](https://mlsanity.com/paper/2308.11953/) (2023)
- [Parallel Split Learning with Global Sampling](https://mlsanity.com/paper/2407.15738/) (2024)
- [CRSFL: Cluster-based Resource-aware Split Federated Learning for Continuous Authentication](https://mlsanity.com/paper/2405.07174/) (2024)
- [Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMix](https://mlsanity.com/paper/2408.01040/) (2024)
- [Data Heterogeneity and Forgotten Labels in Split Federated Learning](https://mlsanity.com/paper/2511.09736/) (2025)
- [A Novel Collaborative Framework for Efficient Synchronization in Split Federated Learning over Wireless Networks](https://mlsanity.com/paper/2503.15559/) (2025)
- [RockNet: Distributed Learning on Ultra-Low-Power Devices](https://mlsanity.com/paper/2510.13320/) (2025)
- [Distributed Machine Learning in D2D-Enabled Heterogeneous Networks: Architectures, Performance, and Open Challenges](https://mlsanity.com/paper/2206.01906/) (2022)
- [When Computing Power Network Meets Distributed Machine Learning: An Efficient Federated Split Learning Framework](https://mlsanity.com/paper/2305.12979/) (2023)
- [Federated Split Learning with Only Positive Labels for resource-constrained IoT environment](https://mlsanity.com/paper/2307.13266/) (2023)
- [Enabling Intelligent Vehicular Networks Through Distributed Learning in the Non-Terrestrial Networks 6G Vision](https://mlsanity.com/paper/2310.05899/) (2023)
- [Modular Distributed Nonconvex Learning with Error Feedback](https://mlsanity.com/paper/2503.14055/) (2025)
- [Makespan Minimization in Split Learning: From Theory to Practice](https://mlsanity.com/paper/2602.06693/) (2026)

## Abstract (arXiv)

Federated learning (FL) and split learning (SL) are state-of-the-art distributed machine learning techniques to enable machine learning training without accessing raw data on clients or end devices. However, their comparative training performance under real-world resource-restricted Internet of Things (IoT) device settings, e.g., Raspberry Pi, remains barely studied, which, to our knowledge, have not yet been evaluated and compared, rendering inconvenient reference for practitioners. This work firstly provides empirical comparisons of FL and SL in real-world IoT settings regarding (i) learning performance with heterogeneous data distributions and (ii) on-device execution overhead. Our analyses in this work demonstrate that the learning performance of SL is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. Recently, FL and SL are combined to form splitfed learning (SFL) to leverage each of their benefits (e.g., parallel training of FL and lightweight on-device computation requirement of SL). This work then considers FL, SL, and SFL, and mount them on Raspberry Pi devices to evaluate their performance, including training time, communication overhead, power consumption, and memory usage. Besides evaluations, we apply two optimizations. Firstly, we generalize SFL by carefully examining the possibility of a hybrid type of model training at the server-side. The generalized SFL merges sequential (dependent) and parallel (independent) processes of model training and is thus beneficial for a system with large-scaled IoT devices, specifically at the server-side operations. Secondly, we propose pragmatic techniques to substantially reduce the communication overhead by up to four times for the SL and (generalized) SFL.