# When MiniBatch SGD Meets SplitFed Learning:Convergence Analysis and Performance Evaluation

arXiv: https://arxiv.org/abs/2308.11953 · HTML: https://mlsanity.com/paper/2308.11953/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 13 of 13)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [SGD: General Analysis and Improved Rates](https://mlsanity.com/paper/1901.09401/) (2019)
- [FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction](https://mlsanity.com/paper/2203.11751/) (2022)
- [Minibatch vs Local SGD for Heterogeneous Distributed Learning](https://mlsanity.com/paper/2006.04735/) (2020)
- [Federated Learning on Non-IID Graphs via Structural Knowledge Sharing](https://mlsanity.com/paper/2211.13009/) (2022)
- [Evaluation and Optimization of Distributed Machine Learning Techniques for Internet of Things](https://mlsanity.com/paper/2103.02762/) (2021)
- [SplitGP: Achieving Both Generalization and Personalization in Federated Learning](https://mlsanity.com/paper/2212.08343/) (2022)
- [Optimal Rate Adaption in Federated Learning with Compressed Communications](https://mlsanity.com/paper/2112.06694/) (2021)
- [Security Analysis of SplitFed Learning](https://mlsanity.com/paper/2212.01716/) (2022)

## Built on by (most cited first, 4 of 4)

- [SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustments](https://mlsanity.com/paper/2405.04875/) (2024)
- [Heterogeneous Federated Learning with Splited Language Model](https://mlsanity.com/paper/2403.16050/) (2024)
- [How Can Incentives and Cut Layer Selection Influence Data Contribution in Split Federated Learning?](https://mlsanity.com/paper/2412.07813/) (2024)
- [A Survey on Split Learning for LLM Fine-Tuning: Models, Systems, and Privacy Optimizations](https://mlsanity.com/paper/2604.24468/) (2026)

## Abstract (arXiv)

Federated learning (FL) enables collaborative model training across distributed clients (e.g., edge devices) without sharing raw data. Yet, FL can be computationally expensive as the clients need to train the entire model multiple times. SplitFed learning (SFL) is a recent distributed approach that alleviates computation workload at the client device by splitting the model at a cut layer into two parts, where clients only need to train part of the model. However, SFL still suffers from the textitclient drift problem when clients' data are highly non-IID. To address this issue, we propose MiniBatch-SFL. This algorithm incorporates MiniBatch SGD into SFL, where the clients train the client-side model in an FL fashion while the server trains the server-side model similar to MiniBatch SGD. We analyze the convergence of MiniBatch-SFL and show that the bound of the expected loss can be obtained by analyzing the expected server-side and client-side model updates, respectively. The server-side updates do not depend on the non-IID degree of the clients' datasets and can potentially mitigate client drift. However, the client-side model relies on the non-IID degree and can be optimized by properly choosing the cut layer. Perhaps counter-intuitive, our empirical result shows that a latter position of the cut layer leads to a smaller average gradient divergence and a better algorithm performance. Moreover, numerical results show that MiniBatch-SFL achieves higher accuracy than conventional SFL and FL. The accuracy improvement can be up to 24.1% and 17.1% with highly non-IID data, respectively.