# Training on Fake Labels: Mitigating Label Leakage in Split Learning via Secure Dimension Transformation

arXiv: https://arxiv.org/abs/2410.09125 · HTML: https://mlsanity.com/paper/2410.09125/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Builds on (most cited first, 24 of 24)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping](https://mlsanity.com/paper/2012.13995/) (2020)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity](https://mlsanity.com/paper/1811.12469/) (2018)
- [Distributed Differential Privacy via Shuffling](https://mlsanity.com/paper/1808.01394/) (2018)
- [Label Leakage and Protection in Two-party Split Learning](https://mlsanity.com/paper/2102.08504/) (2021)
- [Detailed comparison of communication efficiency of split learning and federated learning](https://mlsanity.com/paper/1909.09145/) (2019)
- [CAFE: Catastrophic Data Leakage in Vertical Federated Learning](https://mlsanity.com/paper/2110.15122/) (2021)
- [Split Learning for collaborative deep learning in healthcare](https://mlsanity.com/paper/1912.12115/) (2019)
- [Split Learning in 6G Edge Networks](https://mlsanity.com/paper/2306.12194/) (2023)
- [Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?](https://mlsanity.com/paper/2003.12365/) (2020)
- [Advancements of federated learning towards privacy preservation: from federated learning to split learning](https://mlsanity.com/paper/2011.14818/) (2020)
- [No Peek: A Survey of private distributed deep learning](https://mlsanity.com/paper/1812.03288/) (2018)
- [Distributed Training of Deep Learning Models: A Taxonomic Perspective](https://mlsanity.com/paper/2007.03970/) (2020)
- [FairVFL: A Fair Vertical Federated Learning Framework with Contrastive Adversarial Learning](https://mlsanity.com/paper/2206.03200/) (2022)
- [Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction](https://mlsanity.com/paper/2003.00645/) (2020)
- [Batch Label Inference and Replacement Attacks in Black-Boxed Vertical Federated Learning](https://mlsanity.com/paper/2112.05409/) (2021)
- [Split HE: Fast Secure Inference Combining Split Learning and Homomorphic Encryption](https://mlsanity.com/paper/2202.13351/) (2022)
- [One Pixel Image and RF Signal Based Split Learning for mmWave Received Power Prediction](https://mlsanity.com/paper/1911.01682/) (2019)
- [MISA: Unveiling the Vulnerabilities in Split Federated Learning](https://mlsanity.com/paper/2312.11026/) (2023)

## Built on by (most cited first, 2 of 2)

- [Quantized-Tinyllava: a new multimodal foundation model enables efficient split learning](https://mlsanity.com/paper/2511.23402/) (2025)
- [Oops!... They Stole it Again: Attacks on Split Learning](https://mlsanity.com/paper/2508.10598/) (2025)

## Abstract (arXiv)

Two-party split learning has emerged as a popular paradigm for vertical federated learning. To preserve the privacy of the label owner, split learning utilizes a split model, which only requires the exchange of intermediate representations (IRs) based on the inputs and gradients for each IR between two parties during the learning process. However, split learning has recently been proven to survive label inference attacks. Though several defense methods could be adopted, they either have limited defensive performance or significantly negatively impact the original mission. In this paper, we propose a novel two-party split learning method to defend against existing label inference attacks while maintaining the high utility of the learned models. Specifically, we first craft a dimension transformation module, SecDT, which could achieve bidirectional mapping between original labels and increased K-class labels to mitigate label leakage from the directional perspective. Then, a gradient normalization algorithm is designed to remove the magnitude divergence of gradients from different classes. We propose a softmax-normalized Gaussian noise to mitigate privacy leakage and make our K unknowable to adversaries. We conducted experiments on real-world datasets, including two binary-classification datasets (Avazu and Criteo) and three multi-classification datasets (MNIST, FashionMNIST, CIFAR-10); we also considered current attack schemes, including direction, norm, spectral, and model completion attacks. The detailed experiments demonstrate our proposed method's effectiveness and superiority over existing approaches. For instance, on the Avazu dataset, the attack AUC of evaluated four prominent attacks could be reduced by 0.4532+-0.0127.