# Bagging Provides Assumption-free Stability

arXiv: https://arxiv.org/abs/2301.12600 · HTML: https://mlsanity.com/paper/2301.12600/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 52 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 14 of 14)

- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Stability](https://mlsanity.com/paper/1310.0150/) (2013)
- [Privacy-preserving Prediction](https://mlsanity.com/paper/1803.10266/) (2018)
- [The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression](https://mlsanity.com/paper/1804.09753/) (2018)
- [Iterative Random Forests to detect predictive and stable high-order interactions](https://mlsanity.com/paper/1706.08457/) (2017)
- [Bagging is an Optimal PAC Learner](https://mlsanity.com/paper/2212.02264/) (2022)
- [The Implicit Regularization of Ordinary Least Squares Ensembles](https://mlsanity.com/paper/1910.04743/) (2019)
- [Stable Conformal Prediction Sets](https://mlsanity.com/paper/2112.10224/) (2021)
- [Black-box tests for algorithmic stability](https://mlsanity.com/paper/2111.15546/) (2021)
- [Debiased Machine Learning without Sample-Splitting for Stable Estimators](https://mlsanity.com/paper/2206.01825/) (2022)
- [Bagging in overparameterized learning: Risk characterization and risk monotonization](https://mlsanity.com/paper/2210.11445/) (2022)
- [Derandomizing Knockoffs](https://mlsanity.com/paper/2012.02717/) (2020)
- [Asymptotics of the Sketched Pseudoinverse](https://mlsanity.com/paper/2211.03751/) (2022)

## Built on by (most cited first, 19 of 19)

- [Building a stable classifier with the inflated argmax](https://mlsanity.com/paper/2405.14064/) (2024)
- [Stability of Random Forests and Coverage of Random-Forest Prediction Intervals](https://mlsanity.com/paper/2310.18814/) (2023)
- [Leave a Window Out: Modifying the Jackknife for Predictive Inference in Time Series](https://mlsanity.com/paper/2605.30292/) (2026)
- [The Limits of Assumption-free Tests for Algorithm Performance](https://mlsanity.com/paper/2402.07388/) (2024)
- [Stabilizing black-box model selection with the inflated argmax](https://mlsanity.com/paper/2410.18268/) (2024)
- [Leave-One-Out Stable Conformal Prediction](https://mlsanity.com/paper/2504.12189/) (2025)
- [Assumption-free stability for ranking problems](https://mlsanity.com/paper/2506.02257/) (2025)
- [When does Subagging Work?](https://mlsanity.com/paper/2404.01832/) (2024)
- [Is Algorithmic Stability Testable? A Unified Framework under Computational Constraints](https://mlsanity.com/paper/2405.15107/) (2024)
- [Stability Selection via Variable Decorrelation](https://mlsanity.com/paper/2505.20864/) (2025)
- [Improving reproducibility by controlling random seed stability in machine learning based estimation via bagging](https://mlsanity.com/paper/2604.17694/) (2026)
- [Early wind turbine alarm prediction based on machine learning: AlarmForecasting](https://mlsanity.com/paper/2510.06831/) (2025)
- [A Stable Lasso](https://mlsanity.com/paper/2511.02306/) (2025)
- [Weighted Conformal Prediction Provides Adaptive and Valid Mask-Conditional Coverage for General Missing Data Mechanisms](https://mlsanity.com/paper/2512.14221/) (2025)
- [2D Stability Selection: Design Jittering for Doubly Stable Feature Selection](https://mlsanity.com/paper/2605.02205/) (2026)
- [Stabilizing black-box algorithms through task-oriented randomization](https://mlsanity.com/paper/2606.25269/) (2026)
- [Algorithmic stability via ensembling](https://mlsanity.com/paper/2609.10428/) (2026)
- [A Comprehensive View of Fairness through Distributional Stability](https://mlsanity.com/paper/2609.37061/) (2026)
- [Certification-Enhanced Generalization Bounds](https://mlsanity.com/paper/2610.06238/) (2026)

## Abstract (arXiv)

Bagging is an important technique for stabilizing machine learning models. In this paper, we derive a finite-sample guarantee on the stability of bagging for any model. Our result places no assumptions on the distribution of the data, on the properties of the base algorithm, or on the dimensionality of the covariates. Our guarantee applies to many variants of bagging and is optimal up to a constant. Empirical results validate our findings, showing that bagging successfully stabilizes even highly unstable base algorithms.