# Privacy-Preserving Gradient Boosting Decision Trees

arXiv: https://arxiv.org/abs/1911.04209 · HTML: https://mlsanity.com/paper/1911.04209/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 20 of 20)

- [How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy](https://mlsanity.com/paper/2303.00654/) (2023)
- [Federated Learning for Healthcare Domain - Pipeline, Applications and Challenges](https://mlsanity.com/paper/2211.07893/) (2022)
- [Differentially Private Federated Bayesian Optimization with Distributed Exploration](https://mlsanity.com/paper/2110.14153/) (2021)
- [Accuracy, Interpretability, and Differential Privacy via Explainable Boosting](https://mlsanity.com/paper/2106.09680/) (2021)
- [OpBoost: A Vertical Federated Tree Boosting Framework Based on Order-Preserving Desensitization](https://mlsanity.com/paper/2210.01318/) (2022)
- [FedXGBoost: Privacy-Preserving XGBoost for Federated Learning](https://mlsanity.com/paper/2106.10662/) (2021)
- [Federated Boosted Decision Trees with Differential Privacy](https://mlsanity.com/paper/2210.02910/) (2022)
- [Efficient Batch Homomorphic Encryption for Vertically Federated XGBoost](https://mlsanity.com/paper/2112.04261/) (2021)
- [Improved Differentially Private Regression via Gradient Boosting](https://mlsanity.com/paper/2303.03451/) (2023)
- [SoK: Privacy-Preserving Collaborative Tree-based Model Learning](https://mlsanity.com/paper/2103.08987/) (2021)
- [Classification with Partially Private Features](https://mlsanity.com/paper/2312.07583/) (2023)
- [DP-XGBoost: Private Machine Learning at Scale](https://mlsanity.com/paper/2110.12770/) (2021)
- [Private Boosted Decision Trees via Smooth Re-Weighting](https://mlsanity.com/paper/2201.12648/) (2022)
- [DPSNN: A Differentially Private Spiking Neural Network with Temporal Enhanced Pooling](https://mlsanity.com/paper/2205.12718/) (2022)
- [DPack: Efficiency-Oriented Privacy Budget Scheduling](https://mlsanity.com/paper/2212.13228/) (2022)
- [Differentially-Private Decision Trees and Provable Robustness to Data Poisoning](https://mlsanity.com/paper/2305.15394/) (2023)
- [Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence](https://mlsanity.com/paper/2407.14710/) (2024)
- [Survey of Trustworthy AI: A Meta Decision of AI](https://mlsanity.com/paper/2306.00380/) (2023)
- [S-BDT: Distributed Differentially Private Boosted Decision Trees](https://mlsanity.com/paper/2309.12041/) (2023)
- [Training Set Reconstruction from Differentially Private Forests: How Effective is DP?](https://mlsanity.com/paper/2502.05307/) (2025)

## Abstract (arXiv)

The Gradient Boosting Decision Tree (GBDT) is a popular machine learning model for various tasks in recent years. In this paper, we study how to improve model accuracy of GBDT while preserving the strong guarantee of differential privacy. Sensitivity and privacy budget are two key design aspects for the effectiveness of differential private models. Existing solutions for GBDT with differential privacy suffer from the significant accuracy loss due to too loose sensitivity bounds and ineffective privacy budget allocations (especially across different trees in the GBDT model). Loose sensitivity bounds lead to more noise to obtain a fixed privacy level. Ineffective privacy budget allocations worsen the accuracy loss especially when the number of trees is large. Therefore, we propose a new GBDT training algorithm that achieves tighter sensitivity bounds and more effective noise allocations. Specifically, by investigating the property of gradient and the contribution of each tree in GBDTs, we propose to adaptively control the gradients of training data for each iteration and leaf node clipping in order to tighten the sensitivity bounds. Furthermore, we design a novel boosting framework to allocate the privacy budget between trees so that the accuracy loss can be further reduced. Our experiments show that our approach can achieve much better model accuracy than other baselines.