# Vulnerability Due to Training Order in Split Learning

arXiv: https://arxiv.org/abs/2103.14291 · HTML: https://mlsanity.com/paper/2103.14291/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [Privacy-preserving Federated Brain Tumour Segmentation](https://mlsanity.com/paper/1910.00962/) (2019)
- [Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation](https://mlsanity.com/paper/1810.04304/) (2018)
- [A generic framework for privacy preserving deep learning](https://mlsanity.com/paper/1811.04017/) (2018)
- [Detailed comparison of communication efficiency of split learning and federated learning](https://mlsanity.com/paper/1909.09145/) (2019)
- [Split Learning for collaborative deep learning in healthcare](https://mlsanity.com/paper/1912.12115/) (2019)
- [End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things](https://mlsanity.com/paper/2003.13376/) (2020)
- [Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?](https://mlsanity.com/paper/2003.12365/) (2020)
- [No Peek: A Survey of private distributed deep learning](https://mlsanity.com/paper/1812.03288/) (2018)
- [Comparison of Privacy-Preserving Distributed Deep Learning Methods in Healthcare](https://mlsanity.com/paper/2012.12591/) (2020)
- [Automated Pancreas Segmentation Using Multi-institutional Collaborative Deep Learning](https://mlsanity.com/paper/2009.13148/) (2020)

## Built on by (most cited first, 2 of 2)

- [AdaSplit: Adaptive Trade-offs for Resource-constrained Distributed Deep Learning](https://mlsanity.com/paper/2112.01637/) (2021)
- [Data Heterogeneity and Forgotten Labels in Split Federated Learning](https://mlsanity.com/paper/2511.09736/) (2025)

## Abstract (arXiv)

Split learning (SL) is a privacy-preserving distributed deep learning method used to train a collaborative model without the need for sharing of patient's raw data between clients. In split learning, an additional privacy-preserving algorithm called no-peek algorithm can be incorporated, which is robust to adversarial attacks. The privacy benefits offered by split learning make it suitable for practice in the healthcare domain. However, the split learning algorithm is flawed as the collaborative model is trained sequentially, i.e., one client trains after the other. We point out that the model trained using the split learning algorithm gets biased towards the data of the clients used for training towards the end of a round. This makes SL algorithms highly susceptible to the order in which clients are considered for training. We demonstrate that the model trained using the data of all clients does not perform well on the client's data which was considered earliest in a round for training the model. Moreover, we show that this effect becomes more prominent with the increase in the number of clients. We also demonstrate that the SplitFedv3 algorithm mitigates this problem while still leveraging the privacy benefits provided by split learning.