# Protecting Split Learning by Potential Energy Loss

arXiv: https://arxiv.org/abs/2210.09617 · HTML: https://mlsanity.com/paper/2210.09617/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates](https://mlsanity.com/paper/1911.02151/) (2019)
- [Feature Inference Attack on Model Predictions in Vertical Federated Learning](https://mlsanity.com/paper/2010.10152/) (2020)
- [Unleashing the Tiger: Inference Attacks on Split Learning](https://mlsanity.com/paper/2012.02670/) (2020)
- [How much does your data exploration overfit? Controlling bias via information usage](https://mlsanity.com/paper/1511.05219/) (2015)
- [Label Leakage and Protection in Two-party Split Learning](https://mlsanity.com/paper/2102.08504/) (2021)
- [Generalization Error Bounds for Noisy, Iterative Algorithms](https://mlsanity.com/paper/1801.04295/) (2018)
- [CrypTFlow2: Practical 2-Party Secure Inference](https://mlsanity.com/paper/2010.06457/) (2020)
- [NoPeek: Information leakage reduction to share activations in distributed deep learning](https://mlsanity.com/paper/2008.09161/) (2020)
- [Vertically Federated Graph Neural Network for Privacy-Preserving Node Classification](https://mlsanity.com/paper/2005.11903/) (2020)
- [UnSplit: Data-Oblivious Model Inversion, Model Stealing, and Label Inference Attacks Against Split Learning](https://mlsanity.com/paper/2108.09033/) (2021)
- [Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?](https://mlsanity.com/paper/2003.12365/) (2020)
- [BlindFL: Vertical Federated Machine Learning without Peeking into Your Data](https://mlsanity.com/paper/2206.07975/) (2022)
- [Label Leakage and Protection from Forward Embedding in Vertical Federated Learning](https://mlsanity.com/paper/2203.01451/) (2022)
- [Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness](https://mlsanity.com/paper/1905.11427/) (2019)
- [Differential Private Knowledge Transfer for Privacy-Preserving Cross-Domain Recommendation](https://mlsanity.com/paper/2202.04893/) (2022)
- [Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction](https://mlsanity.com/paper/2003.00645/) (2020)
- [DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples](https://mlsanity.com/paper/2110.13740/) (2021)
- [ExPLoit: Extracting Private Labels in Split Learning](https://mlsanity.com/paper/2112.01299/) (2021)
- [Spatio-Temporal Split Learning for Privacy-Preserving Medical Platforms: Case Studies with COVID-19 CT, X-Ray, and Cholesterol Data](https://mlsanity.com/paper/2108.10147/) (2021)
- [Maximum Margin Multiclass Nearest Neighbors](https://mlsanity.com/paper/1401.7898/) (2014)

## Built on by (most cited first, 3 of 3)

- [A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective](https://mlsanity.com/paper/2402.03688/) (2024)
- [Secure Vertical Federated Learning Under Unreliable Connectivity](https://mlsanity.com/paper/2305.16794/) (2023)
- [Defending Label Inference Attacks in Split Learning under Regression Setting](https://mlsanity.com/paper/2308.09448/) (2023)

## Abstract (arXiv)

As a practical privacy-preserving learning method, split learning has drawn much attention in academia and industry. However, its security is constantly being questioned since the intermediate results are shared during training and inference. In this paper, we focus on the privacy leakage from the forward embeddings of split learning. Specifically, since the forward embeddings contain too much information about the label, the attacker can either use a few labeled samples to fine-tune the top model or perform unsupervised attacks such as clustering to infer the true labels from the forward embeddings. To prevent such kind of privacy leakage, we propose the potential energy loss to make the forward embeddings become more 'complicated', by pushing embeddings of the same class towards the decision boundary. Therefore, it is hard for the attacker to learn from the forward embeddings. Experiment results show that our method significantly lowers the performance of both fine-tuning attacks and clustering attacks.