# Feature Bagging Provides Stability

arXiv: https://arxiv.org/abs/2607.26964 · HTML: https://mlsanity.com/paper/2607.26964/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 64 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Builds on (most cited first, 5 of 5)

- [Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers](https://mlsanity.com/paper/2402.01502/) (2024)
- [Stabilizing black-box model selection with the inflated argmax](https://mlsanity.com/paper/2410.18268/) (2024)
- [Assumption-free stability for ranking problems](https://mlsanity.com/paper/2506.02257/) (2025)
- [Adaptive Split Balancing for Optimal Random Forest](https://mlsanity.com/paper/2402.11228/) (2024)
- [The Exact Risks of Reference Panel-based Regularized Estimators](https://mlsanity.com/paper/2401.11359/) (2024)

## Abstract (arXiv)

We study feature bagging through the lens of algorithmic stability. Feature bagging is an ensemble strategy that aggregates base learners trained on randomly subsampled feature subsets, possibly in a data-dependent manner. We introduce feature instability (FI), the feature-axis analogue of instance instability (II), which measures sensitivity to removing a single feature. Smaller values of II or FI correspond to stronger stability, and our experiments show that FI captures generalization-relevant information complementary to II. Within this framework, we analyze feature bagging in both a parametric linear model and a model-free setting inspired by recursive feature subsampling in random forests. In both settings, we establish formal guarantees showing that feature bagging improves the relevant stability relative to its non-bagged counterpart, with larger improvements under more aggressive subsampling. We further show that a modest number of bagging rounds is sufficient to approach the infinite-bagging stability level.