# S-BDT: Distributed Differentially Private Boosted Decision Trees

arXiv: https://arxiv.org/abs/2309.12041 · HTML: https://mlsanity.com/paper/2309.12041/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 14 of 14)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences](https://mlsanity.com/paper/1807.01647/) (2018)
- [The Discrete Gaussian for Differential Privacy](https://mlsanity.com/paper/2004.00010/) (2020)
- [Hypothesis Testing Interpretations and Renyi Differential Privacy](https://mlsanity.com/paper/1905.09982/) (2019)
- [Individual Privacy Accounting via a Renyi Filter](https://mlsanity.com/paper/2008.11193/) (2020)
- [Accuracy, Interpretability, and Differential Privacy via Explainable Boosting](https://mlsanity.com/paper/2106.09680/) (2021)
- [Individual Privacy Accounting with Gaussian Differential Privacy](https://mlsanity.com/paper/2209.15596/) (2022)
- [Privacy-Preserving Gradient Boosting Decision Trees](https://mlsanity.com/paper/1911.04209/) (2019)
- [Differentially Private Random Decision Forests using Smooth Sensitivity](https://mlsanity.com/paper/1606.03572/) (2016)
- [Federated Boosted Decision Trees with Differential Privacy](https://mlsanity.com/paper/2210.02910/) (2022)
- [MVG Mechanism: Differential Privacy under Matrix-Valued Query](https://mlsanity.com/paper/1801.00823/) (2018)
- [Differentially- and non-differentially-private random decision trees](https://mlsanity.com/paper/1410.6973/) (2014)

## Abstract (arXiv)

We introduce S-BDT: a novel (varepsilon,δ)-differentially private distributed gradient boosted decision tree (GBDT) learner that improves the protection of single training data points (privacy) while achieving meaningful learning goals, such as accuracy or regression error (utility). S-BDT uses less noise by relying on non-spherical multivariate Gaussian noise, for which we show tight subsampling bounds for privacy amplification and incorporate that into a R'enyi filter for individual privacy accounting. We experimentally reach the same utility while saving 50% in terms of epsilon for varepsilon ≤ 0.5 on the Abalone regression dataset (dataset size ≈ 4K), saving 30% in terms of epsilon for varepsilon ≤ 0.08 for the Adult classification dataset (dataset size ≈ 50K), and saving 30% in terms of epsilon for varepsilon≤0.03 for the Spambase classification dataset (dataset size ≈ 5K). Moreover, we show that for situations where a GBDT is learning a stream of data that originates from different subpopulations (non-IID), S-BDT improves the saving of epsilon even further.