# Label Leakage and Protection from Forward Embedding in Vertical Federated Learning

arXiv: https://arxiv.org/abs/2203.01451 · HTML: https://mlsanity.com/paper/2203.01451/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity](https://mlsanity.com/paper/1811.12469/) (2018)
- [Deep Learning with Label Differential Privacy](https://mlsanity.com/paper/2102.06062/) (2021)
- [Distributed Differential Privacy via Shuffling](https://mlsanity.com/paper/1808.01394/) (2018)
- [Label Leakage and Protection in Two-party Split Learning](https://mlsanity.com/paper/2102.08504/) (2021)
- [No Peek: A Survey of private distributed deep learning](https://mlsanity.com/paper/1812.03288/) (2018)
- [Asymmetrical Vertical Federated Learning](https://mlsanity.com/paper/2004.07427/) (2020)
- [Defending against Reconstruction Attack in Vertical Federated Learning](https://mlsanity.com/paper/2107.09898/) (2021)
- [A fast algorithm for computing distance correlation](https://mlsanity.com/paper/1810.11332/) (2018)
- [A Statistically and Numerically Efficient Independence Test based on Random Projections and Distance Covariance](https://mlsanity.com/paper/1701.06054/) (2017)

## Built on by (most cited first, 27 of 27)

- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation](https://mlsanity.com/paper/2310.00339/) (2023)
- [Vertical Federated Learning: A Structured Literature Review](https://mlsanity.com/paper/2212.00622/) (2022)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [Communication-Efficient Vertical Federated Learning with Limited Overlapping Samples](https://mlsanity.com/paper/2303.16270/) (2023)
- [Vertical Semi-Federated Learning for Efficient Online Advertising](https://mlsanity.com/paper/2209.15635/) (2022)
- [HashVFL: Defending Against Data Reconstruction Attacks in Vertical Federated Learning](https://mlsanity.com/paper/2212.00325/) (2022)
- [Vertical Federated Learning: Taxonomies, Threats, and Prospects](https://mlsanity.com/paper/2302.01550/) (2023)
- [VFLAIR: A Research Library and Benchmark for Vertical Federated Learning](https://mlsanity.com/paper/2310.09827/) (2023)
- [A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective](https://mlsanity.com/paper/2402.03688/) (2024)
- [FedAds: A Benchmark for Privacy-Preserving CVR Estimation with Vertical Federated Learning](https://mlsanity.com/paper/2305.08328/) (2023)
- [Eliminating Label Leakage in Tree-Based Vertical Federated Learning](https://mlsanity.com/paper/2307.10318/) (2023)
- [Online Vertical Federated Learning for Cooperative Spectrum Sensing](https://mlsanity.com/paper/2312.11363/) (2023)
- [VFed-SSD: Towards Practical Vertical Federated Advertising](https://mlsanity.com/paper/2205.15987/) (2022)
- [Secure Vertical Federated Learning Under Unreliable Connectivity](https://mlsanity.com/paper/2305.16794/) (2023)
- [Protecting Split Learning by Potential Energy Loss](https://mlsanity.com/paper/2210.09617/) (2022)
- [Secure and Fast Asynchronous Vertical Federated Learning via Cascaded Hybrid Optimization](https://mlsanity.com/paper/2306.16077/) (2023)
- [Defending Label Inference Attacks in Split Learning under Regression Setting](https://mlsanity.com/paper/2308.09448/) (2023)
- [LabObf: A Label Protection Scheme for Vertical Federated Learning Through Label Obfuscation](https://mlsanity.com/paper/2405.17042/) (2024)
- [Just a Simple Transformation is Enough for Data Protection in Vertical Federated Learning](https://mlsanity.com/paper/2412.11689/) (2024)
- [GAN-based Vertical Federated Learning for Label Protection in Binary Classification](https://mlsanity.com/paper/2302.02245/) (2023)
- [Survey of Privacy Threats and Countermeasures in Federated Learning](https://mlsanity.com/paper/2402.00342/) (2024)
- [Multimodal Federated Learning: A Survey through the Lens of Different FL Paradigms](https://mlsanity.com/paper/2505.21792/) (2025)
- [HASSLE: A Self-Supervised Learning Enhanced Hijacking Attack on Vertical Federated Learning](https://mlsanity.com/paper/2507.10162/) (2025)
- [VMask: Tunable Label Privacy Protection for Vertical Federated Learning via Layer Masking](https://mlsanity.com/paper/2507.14629/) (2025)
- [Private Vertical Federated Inference for Time-Series](https://mlsanity.com/paper/2605.08343/) (2026)

## Abstract (arXiv)

Vertical federated learning (vFL) has gained much attention and been deployed to solve machine learning problems with data privacy concerns in recent years. However, some recent work demonstrated that vFL is vulnerable to privacy leakage even though only the forward intermediate embedding (rather than raw features) and backpropagated gradients (rather than raw labels) are communicated between the involved participants. As the raw labels often contain highly sensitive information, some recent work has been proposed to prevent the label leakage from the backpropagated gradients effectively in vFL. However, these work only identified and defended the threat of label leakage from the backpropagated gradients. None of these work has paid attention to the problem of label leakage from the intermediate embedding. In this paper, we propose a practical label inference method which can steal private labels effectively from the shared intermediate embedding even though some existing protection methods such as label differential privacy and gradients perturbation are applied. The effectiveness of the label attack is inseparable from the correlation between the intermediate embedding and corresponding private labels. To mitigate the issue of label leakage from the forward embedding, we add an additional optimization goal at the label party to limit the label stealing ability of the adversary by minimizing the distance correlation between the intermediate embedding and corresponding private labels. We conducted massive experiments to demonstrate the effectiveness of our proposed protection methods.