# Label Leakage and Protection in Two-party Split Learning

arXiv: https://arxiv.org/abs/2102.08504 · HTML: https://mlsanity.com/paper/2102.08504/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 17 Feb 2021.

## Builds on (most cited first, 20 of 20)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Differentially Private Federated Learning: A Client Level Perspective](https://mlsanity.com/paper/1712.07557/) (2017)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [Differentially Private Learning with Adaptive Clipping](https://mlsanity.com/paper/1905.03871/) (2019)
- [Threats to Federated Learning: A Survey](https://mlsanity.com/paper/2003.02133/) (2020)
- [Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity](https://mlsanity.com/paper/1811.12469/) (2018)
- [Protection Against Reconstruction and Its Applications in Private Federated Learning](https://mlsanity.com/paper/1812.00984/) (2018)
- [Deep Learning with Label Differential Privacy](https://mlsanity.com/paper/2102.06062/) (2021)
- [Distributed Differential Privacy via Shuffling](https://mlsanity.com/paper/1808.01394/) (2018)
- [Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware](https://mlsanity.com/paper/1806.03287/) (2018)
- [A General Approach to Adding Differential Privacy to Iterative Training Procedures](https://mlsanity.com/paper/1812.06210/) (2018)
- [AdaCliP: Adaptive Clipping for Private SGD](https://mlsanity.com/paper/1908.07643/) (2019)
- [QUOTIENT: Two-Party Secure Neural Network Training and Prediction](https://mlsanity.com/paper/1907.03372/) (2019)
- [A Review of Homomorphic Encryption Libraries for Secure Computation](https://mlsanity.com/paper/1812.02428/) (2018)

## Built on by (most cited first, 30 of 90)

- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges](https://mlsanity.com/paper/2201.08135/) (2022)
- [UnSplit: Data-Oblivious Model Inversion, Model Stealing, and Label Inference Attacks Against Split Learning](https://mlsanity.com/paper/2108.09033/) (2021)
- [BlindFL: Vertical Federated Machine Learning without Peeking into Your Data](https://mlsanity.com/paper/2206.07975/) (2022)
- [Trading Off Privacy, Utility and Efficiency in Federated Learning](https://mlsanity.com/paper/2209.00230/) (2022)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [Privacy-preserving Federated Adversarial Domain Adaption over Feature Groups for Interpretability](https://mlsanity.com/paper/2111.10934/) (2021)
- [Label Leakage and Protection from Forward Embedding in Vertical Federated Learning](https://mlsanity.com/paper/2203.01451/) (2022)
- [Differentially Private Label Protection in Split Learning](https://mlsanity.com/paper/2203.02073/) (2022)
- [User-Level Label Leakage from Gradients in Federated Learning](https://mlsanity.com/paper/2105.09369/) (2021)
- [Vertical Federated Learning without Revealing Intersection Membership](https://mlsanity.com/paper/2106.05508/) (2021)
- [Graph-Fraudster: Adversarial Attacks on Graph Neural Network Based Vertical Federated Learning](https://mlsanity.com/paper/2110.06468/) (2021)
- [A Hybrid Self-Supervised Learning Framework for Vertical Federated Learning](https://mlsanity.com/paper/2208.08934/) (2022)
- [Privacy-Aware Document Visual Question Answering](https://mlsanity.com/paper/2312.10108/) (2023)
- [Batch Label Inference and Replacement Attacks in Black-Boxed Vertical Federated Learning](https://mlsanity.com/paper/2112.05409/) (2021)
- [ExPLoit: Extracting Private Labels in Split Learning](https://mlsanity.com/paper/2112.01299/) (2021)
- [Similarity-based Label Inference Attack against Training and Inference of Split Learning](https://mlsanity.com/paper/2203.05222/) (2022)
- [Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates](https://mlsanity.com/paper/2207.14628/) (2022)
- [Regression with Label Differential Privacy](https://mlsanity.com/paper/2212.06074/) (2022)
- [On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook](https://mlsanity.com/paper/2307.16680/) (2023)
- [Combined Federated and Split Learning in Edge Computing for Ubiquitous Intelligence in Internet of Things: State of the Art and Future Directions](https://mlsanity.com/paper/2207.09611/) (2022)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [Feature Reconstruction Attacks and Countermeasures of DNN training in Vertical Federated Learning](https://mlsanity.com/paper/2210.06771/) (2022)
- [Residue-based Label Protection Mechanisms in Vertical Logistic Regression](https://mlsanity.com/paper/2205.04166/) (2022)
- [A Framework for Evaluating Privacy-Utility Trade-off in Vertical Federated Learning](https://mlsanity.com/paper/2209.03885/) (2022)
- [FedPass: Privacy-Preserving Vertical Federated Deep Learning with Adaptive Obfuscation](https://mlsanity.com/paper/2301.12623/) (2023)
- [Client-specific Property Inference against Secure Aggregation in Federated Learning](https://mlsanity.com/paper/2303.03908/) (2023)
- [Passive Inference Attacks on Split Learning via Adversarial Regularization](https://mlsanity.com/paper/2310.10483/) (2023)
- [Label Inference Attack against Split Learning under Regression Setting](https://mlsanity.com/paper/2301.07284/) (2023)
- [Federated Learning of Models Pre-Trained on Different Features with Consensus Graphs](https://mlsanity.com/paper/2306.01240/) (2023)

## Abstract (arXiv)

Two-party split learning is a popular technique for learning a model across feature-partitioned data. In this work, we explore whether it is possible for one party to steal the private label information from the other party during split training, and whether there are methods that can protect against such attacks. Specifically, we first formulate a realistic threat model and propose a privacy loss metric to quantify label leakage in split learning. We then show that there exist two simple yet effective methods within the threat model that can allow one party to accurately recover private ground-truth labels owned by the other party. To combat these attacks, we propose several random perturbation techniques, including textttMarvell, an approach that strategically finds the structure of the noise perturbation by minimizing the amount of label leakage (measured through our quantification metric) of a worst-case adversary. We empirically demonstrate the effectiveness of our protection techniques against the identified attacks, and show that textttMarvell in particular has improved privacy-utility tradeoffs relative to baseline approaches.