# Practical Federated Gradient Boosting Decision Trees

arXiv: https://arxiv.org/abs/1911.04206 · HTML: https://mlsanity.com/paper/1911.04206/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 30 of 54)

- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges](https://mlsanity.com/paper/2211.08413/) (2022)
- [Federated Learning with Label Distribution Skew via Logits Calibration](https://mlsanity.com/paper/2209.00189/) (2022)
- [Decentralized Federated Learning: A Survey and Perspective](https://mlsanity.com/paper/2306.01603/) (2023)
- [Provably Secure Federated Learning against Malicious Clients](https://mlsanity.com/paper/2102.01854/) (2021)
- [Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee](https://mlsanity.com/paper/2110.14074/) (2021)
- [A Survey on Heterogeneous Federated Learning](https://mlsanity.com/paper/2210.04505/) (2022)
- [Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions](https://mlsanity.com/paper/2303.02213/) (2023)
- [Membership Inference Attacks and Defenses in Federated Learning: A Survey](https://mlsanity.com/paper/2412.06157/) (2024)
- [Towards Federated Clustering: A Federated Fuzzy c-Means Algorithm (FFCM)](https://mlsanity.com/paper/2201.07316/) (2022)
- [FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations](https://mlsanity.com/paper/2203.13789/) (2022)
- [A Survey on Decentralized Federated Learning](https://mlsanity.com/paper/2308.04604/) (2023)
- [Decentralized Federated Learning: A Survey on Security and Privacy](https://mlsanity.com/paper/2401.17319/) (2024)
- [Exploiting Label Skews in Federated Learning with Model Concatenation](https://mlsanity.com/paper/2312.06290/) (2023)
- [Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Data](https://mlsanity.com/paper/2206.00686/) (2022)
- [OpBoost: A Vertical Federated Tree Boosting Framework Based on Order-Preserving Desensitization](https://mlsanity.com/paper/2210.01318/) (2022)
- [Gradient-less Federated Gradient Boosting Trees with Learnable Learning Rates](https://mlsanity.com/paper/2304.07537/) (2023)
- [Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning](https://mlsanity.com/paper/2105.09540/) (2021)
- [FedXGBoost: Privacy-Preserving XGBoost for Federated Learning](https://mlsanity.com/paper/2106.10662/) (2021)
- [Enabling SQL-based Training Data Debugging for Federated Learning](https://mlsanity.com/paper/2108.11884/) (2021)
- [Federated Learning Versus Classical Machine Learning: A Convergence Comparison](https://mlsanity.com/paper/2107.10976/) (2021)
- [An Empirical Study of Personalized Federated Learning](https://mlsanity.com/paper/2206.13190/) (2022)
- [FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler](https://mlsanity.com/paper/2405.15458/) (2024)
- [Federated Functional Gradient Boosting](https://mlsanity.com/paper/2103.06972/) (2021)
- [Effective and Efficient Federated Tree Learning on Hybrid Data](https://mlsanity.com/paper/2310.11865/) (2023)
- [An Interpretable Client Decision Tree Aggregation process for Federated Learning](https://mlsanity.com/paper/2404.02510/) (2024)
- [SoK: Privacy-Preserving Collaborative Tree-based Model Learning](https://mlsanity.com/paper/2103.08987/) (2021)
- [A review of Federated Learning in Intrusion Detection Systems for IoT](https://mlsanity.com/paper/2204.12443/) (2022)
- [FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learning](https://mlsanity.com/paper/2206.03852/) (2022)

## Abstract (arXiv)

Gradient Boosting Decision Trees (GBDTs) have become very successful in recent years, with many awards in machine learning and data mining competitions. There have been several recent studies on how to train GBDTs in the federated learning setting. In this paper, we focus on horizontal federated learning, where data samples with the same features are distributed among multiple parties. However, existing studies are not efficient or effective enough for practical use. They suffer either from the inefficiency due to the usage of costly data transformations such as secret sharing and homomorphic encryption, or from the low model accuracy due to differential privacy designs. In this paper, we study a practical federated environment with relaxed privacy constraints. In this environment, a dishonest party might obtain some information about the other parties' data, but it is still impossible for the dishonest party to derive the actual raw data of other parties. Specifically, each party boosts a number of trees by exploiting similarity information based on locality-sensitive hashing. We prove that our framework is secure without exposing the original record to other parties, while the computation overhead in the training process is kept low. Our experimental studies show that, compared with normal training with the local data of each party, our approach can significantly improve the predictive accuracy, and achieve comparable accuracy to the original GBDT with the data from all parties.