# Eliminating Label Leakage in Tree-Based Vertical Federated Learning

arXiv: https://arxiv.org/abs/2307.10318 · HTML: https://mlsanity.com/paper/2307.10318/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 70 references that are in the map.
- First posted on arXiv 19 Jul 2023.

## Builds on (most cited first, 25 of 25)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [Deep Learning with Label Differential Privacy](https://mlsanity.com/paper/2102.06062/) (2021)
- [Label Leakage and Protection in Two-party Split Learning](https://mlsanity.com/paper/2102.08504/) (2021)
- [Privacy Preserving Vertical Federated Learning for Tree-based Models](https://mlsanity.com/paper/2008.06170/) (2020)
- [Private Learning and Sanitization: Pure vs. Approximate Differential Privacy](https://mlsanity.com/paper/1407.2674/) (2014)
- [Improving Robustness to Model Inversion Attacks via Mutual Information Regularization](https://mlsanity.com/paper/2009.05241/) (2020)
- [Federated Forest](https://mlsanity.com/paper/1905.10053/) (2019)
- [Differential Privacy as a Mutual Information Constraint](https://mlsanity.com/paper/1608.03677/) (2016)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [FederBoost: Private Federated Learning for GBDT](https://mlsanity.com/paper/2011.02796/) (2020)
- [Label Leakage and Protection from Forward Embedding in Vertical Federated Learning](https://mlsanity.com/paper/2203.01451/) (2022)
- [Large-Scale Secure XGB for Vertical Federated Learning](https://mlsanity.com/paper/2005.08479/) (2020)
- [A Communication Efficient Collaborative Learning Framework for Distributed Features](https://mlsanity.com/paper/1912.11187/) (2019)
- [User-Level Label Leakage from Gradients in Federated Learning](https://mlsanity.com/paper/2105.09369/) (2021)
- [A Vertical Federated Learning Method for Interpretable Scorecard and Its Application in Credit Scoring](https://mlsanity.com/paper/2009.06218/) (2020)
- [Modelling and Quantifying Membership Information Leakage in Machine Learning](https://mlsanity.com/paper/2001.10648/) (2020)
- [OpBoost: A Vertical Federated Tree Boosting Framework Based on Order-Preserving Desensitization](https://mlsanity.com/paper/2210.01318/) (2022)
- [Residue-based Label Protection Mechanisms in Vertical Logistic Regression](https://mlsanity.com/paper/2205.04166/) (2022)
- [PIVODL: Privacy-preserving vertical federated learning over distributed labels](https://mlsanity.com/paper/2108.11444/) (2021)
- [Enabling SQL-based Training Data Debugging for Federated Learning](https://mlsanity.com/paper/2108.11884/) (2021)
- [An Efficient and Robust System for Vertically Federated Random Forest](https://mlsanity.com/paper/2201.10761/) (2022)
- [Privacy-Preserving Training of Tree Ensembles over Continuous Data](https://mlsanity.com/paper/2106.02769/) (2021)

## Built on by (most cited first, 6 of 6)

- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [VFLAIR: A Research Library and Benchmark for Vertical Federated Learning](https://mlsanity.com/paper/2310.09827/) (2023)
- [A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective](https://mlsanity.com/paper/2402.03688/) (2024)
- [Survey of Privacy Threats and Countermeasures in Federated Learning](https://mlsanity.com/paper/2402.00342/) (2024)
- [Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregation](https://mlsanity.com/paper/2604.07125/) (2026)
- [Practical Anonymous Two-Party Gradient Boosting Decision Tree](https://mlsanity.com/paper/2605.26903/) (2026)

## Abstract (arXiv)

Vertical federated learning (VFL) enables multiple parties with disjoint features of a common user set to train a machine learning model without sharing their private data. Tree-based models have become prevalent in VFL due to their interpretability and efficiency. However, the vulnerability of tree-based VFL has not been sufficiently investigated. In this study, we first introduce a novel label inference attack, ID2Graph, which utilizes the sets of record IDs assigned to each node (i.e., instance space)to deduce private training labels. ID2Graph attack generates a graph structure from training samples, extracts communities from the graph, and clusters the local dataset using community information. To counteract label leakage from the instance space, we propose two effective defense mechanisms, Grafting-LDP, which improves the utility of label differential privacy with post-processing, and andID-LMID, which focuses on mutual information regularization. Comprehensive experiments on various datasets reveal that ID2Graph presents significant risks to tree-based models such as RandomForest and XGBoost. Further evaluations of these benchmarks demonstrate that our defense methods effectively mitigate label leakage in such instances