# Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importance

arXiv: https://arxiv.org/abs/2003.03629 · HTML: https://mlsanity.com/paper/2003.03629/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2020.

## Built on by (most cited first, 10 of 10)

- [Trees, Forests, Chickens, and Eggs: When and Why to Prune Trees in a Random Forest](https://mlsanity.com/paper/2103.16700/) (2021)
- [MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA](https://mlsanity.com/paper/2102.13347/) (2021)
- [Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning](https://mlsanity.com/paper/2102.12328/) (2021)
- [Randomization Can Reduce Both Bias and Variance: A Case Study in Random Forests](https://mlsanity.com/paper/2402.12668/) (2024)
- [Random Forests as Statistical Procedures: Design, Variance, and Dependence](https://mlsanity.com/paper/2602.13104/) (2026)
- [Sequential Permutation Testing of Random Forest Variable Importance Measures](https://mlsanity.com/paper/2206.01284/) (2022)
- [Global Censored Quantile Random Forest](https://mlsanity.com/paper/2410.12209/) (2024)
- [Targeted Learning for Variable Importance](https://mlsanity.com/paper/2411.02221/) (2024)
- [Targeted tuning of random forests for quantile estimation and prediction intervals](https://mlsanity.com/paper/2507.01430/) (2025)
- [Routing Probes Can Improve Without New Information: An Exact-Null Audit of Uncertainty Beyond Model Outputs](https://mlsanity.com/paper/2609.38956/) (2026)

## Abstract (arXiv)

As the size, complexity, and availability of data continues to grow, scientists are increasingly relying upon black-box learning algorithms that can often provide accurate predictions with minimal a priori model specifications. Tools like random forests have an established track record of off-the-shelf success and even offer various strategies for analyzing the underlying relationships among variables. Here, motivated by recent insights into random forest behavior, we introduce the simple idea of augmented bagging (AugBagg), a procedure that operates in an identical fashion to classical bagging and random forests, but which operates on a larger, augmented space containing additional randomly generated noise features. Surprisingly, we demonstrate that this simple act of including extra noise variables in the model can lead to dramatic improvements in out-of-sample predictive accuracy, sometimes outperforming even an optimally tuned traditional random forest. As a result, intuitive notions of variable importance based on improved model accuracy may be deeply flawed, as even purely random noise can routinely register as statistically significant. Numerous demonstrations on both real and synthetic data are provided along with a proposed solution.