# On the Intrinsic Differential Privacy of Bagging

arXiv: https://arxiv.org/abs/2008.09845 · HTML: https://mlsanity.com/paper/2008.09845/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2020.

## Built on by (most cited first, 7 of 7)

- [How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy](https://mlsanity.com/paper/2303.00654/) (2023)
- [Selective Ensembles for Consistent Predictions](https://mlsanity.com/paper/2111.08230/) (2021)
- [Trained Random Forests Completely Reveal your Dataset](https://mlsanity.com/paper/2402.19232/) (2024)
- [Differential Privacy Under Class Imbalance: Methods and Empirical Insights](https://mlsanity.com/paper/2411.05733/) (2024)
- [Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning](https://mlsanity.com/paper/2212.03334/) (2022)
- [Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis](https://mlsanity.com/paper/2209.00439/) (2022)
- [Personalized DP-SGD using Sampling Mechanisms](https://mlsanity.com/paper/2305.15165/) (2023)

## Abstract (arXiv)

Differentially private machine learning trains models while protecting privacy of the sensitive training data. The key to obtain differentially private models is to introduce noise/randomness to the training process. In particular, existing differentially private machine learning methods add noise to the training data, the gradients, the loss function, and/or the model itself. Bagging, a popular ensemble learning framework, randomly creates some subsamples of the training data, trains a base model for each subsample using a base learner, and takes majority vote among the base models when making predictions. Bagging has intrinsic randomness in the training process as it randomly creates subsamples. Our major theoretical results show that such intrinsic randomness already makes Bagging differentially private without the needs of additional noise. In particular, we prove that, for any base learner, Bagging with and without replacement respectively achieves left(N· k · lnfracn+1n,1- (fracn-1n)N· kright)-differential privacy and left(lnfracn+1n+1-N· k, fracN· kn right)-differential privacy, where n is the training data size, k is the subsample size, and N is the number of base models. Moreover, we prove that if no assumptions about the base learner are made, our derived privacy guarantees are tight. We empirically evaluate Bagging on MNIST and CIFAR10. Our experimental results demonstrate that Bagging achieves significantly higher accuracies than state-of-the-art differentially private machine learning methods with the same privacy budgets.