# Federated Boosted Decision Trees with Differential Privacy

arXiv: https://arxiv.org/abs/2210.02910 · HTML: https://mlsanity.com/paper/2210.02910/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 66 references that are in the map.
- First posted on arXiv 6 Oct 2022.

## Builds on (most cited first, 30 of 32)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising](https://mlsanity.com/paper/1805.06530/) (2018)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [Why do tree-based models still outperform deep learning on tabular data?](https://mlsanity.com/paper/2207.08815/) (2022)
- [Numerical Composition of Differential Privacy](https://mlsanity.com/paper/2106.02848/) (2021)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Practical and Private (Deep) Learning without Sampling or Shuffling](https://mlsanity.com/paper/2103.00039/) (2021)
- [The Discrete Gaussian for Differential Privacy](https://mlsanity.com/paper/2004.00010/) (2020)
- [Toward Training at ImageNet Scale with Differential Privacy](https://mlsanity.com/paper/2201.12328/) (2022)
- [The Skellam Mechanism for Differentially Private Federated Learning](https://mlsanity.com/paper/2110.04995/) (2021)
- [Do We Really Need Deep Learning Models for Time Series Forecasting?](https://mlsanity.com/paper/2101.02118/) (2021)
- [Defending against Reconstruction Attacks with R'enyi Differential Privacy](https://mlsanity.com/paper/2202.07623/) (2022)
- [FederBoost: Private Federated Learning for GBDT](https://mlsanity.com/paper/2011.02796/) (2020)
- [Large-Scale Secure XGB for Vertical Federated Learning](https://mlsanity.com/paper/2005.08479/) (2020)
- [Accuracy, Interpretability, and Differential Privacy via Explainable Boosting](https://mlsanity.com/paper/2106.09680/) (2021)
- [Privacy-Preserving Gradient Boosting Decision Trees](https://mlsanity.com/paper/1911.04209/) (2019)
- [SecureGBM: Secure Multi-Party Gradient Boosting](https://mlsanity.com/paper/1911.11997/) (2019)
- [Differentially Private Random Decision Forests using Smooth Sensitivity](https://mlsanity.com/paper/1606.03572/) (2016)
- [Secure Collaborative Training and Inference for XGBoost](https://mlsanity.com/paper/2010.02524/) (2020)
- [FedXGBoost: Privacy-Preserving XGBoost for Federated Learning](https://mlsanity.com/paper/2106.10662/) (2021)
- [Privacy-Preserving XGBoost Inference](https://mlsanity.com/paper/2011.04789/) (2020)

## Built on by (most cited first, 9 of 9)

- [A Survey for Federated Learning Evaluations: Goals and Measures](https://mlsanity.com/paper/2308.11841/) (2023)
- [Cross-silo Federated Learning with Record-level Personalized Differential Privacy](https://mlsanity.com/paper/2401.16251/) (2024)
- [Effective and Efficient Federated Tree Learning on Hybrid Data](https://mlsanity.com/paper/2310.11865/) (2023)
- [Deep incremental learning models for financial temporal tabular datasets with distribution shifts](https://mlsanity.com/paper/2303.07925/) (2023)
- [DPM: Clustering Sensitive Data through Separation](https://mlsanity.com/paper/2307.02969/) (2023)
- [S-BDT: Distributed Differentially Private Boosted Decision Trees](https://mlsanity.com/paper/2309.12041/) (2023)
- [Privacy-preserving federated prediction of pain intensity change based on multi-center survey data](https://mlsanity.com/paper/2409.07997/) (2024)
- [TimberStrike: Dataset Reconstruction Attack Revealing Privacy Leakage in Federated Tree-Based Systems](https://mlsanity.com/paper/2506.07605/) (2025)
- [Towards Explainable Federated Learning: Understanding the Impact of Differential Privacy](https://mlsanity.com/paper/2602.10100/) (2026)

## Abstract (arXiv)

There is great demand for scalable, secure, and efficient privacy-preserving machine learning models that can be trained over distributed data. While deep learning models typically achieve the best results in a centralized non-secure setting, different models can excel when privacy and communication constraints are imposed. Instead, tree-based approaches such as XGBoost have attracted much attention for their high performance and ease of use; in particular, they often achieve state-of-the-art results on tabular data. Consequently, several recent works have focused on translating Gradient Boosted Decision Tree (GBDT) models like XGBoost into federated settings, via cryptographic mechanisms such as Homomorphic Encryption (HE) and Secure Multi-Party Computation (MPC). However, these do not always provide formal privacy guarantees, or consider the full range of hyperparameters and implementation settings. In this work, we implement the GBDT model under Differential Privacy (DP). We propose a general framework that captures and extends existing approaches for differentially private decision trees. Our framework of methods is tailored to the federated setting, and we show that with a careful choice of techniques it is possible to achieve very high utility while maintaining strong levels of privacy.