# ControlBurn: Feature Selection by Sparse Forests

arXiv: https://arxiv.org/abs/2107.00219 · HTML: https://mlsanity.com/paper/2107.00219/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 1 Jul 2021.

## Builds on (most cited first, 12 of 12)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [Best Subset Selection via a Modern Optimization Lens](https://mlsanity.com/paper/1507.03133/) (2015)
- [S-LIME: Stabilized-LIME for Model Explanation](https://mlsanity.com/paper/2106.07875/) (2021)
- [Unbiased Measurement of Feature Importance in Tree-Based Methods](https://mlsanity.com/paper/1903.05179/) (2019)
- [Randomized Sketches of Convex Programs with Sharp Guarantees](https://mlsanity.com/paper/1404.7203/) (2014)
- [Trees, forests, and impurity-based variable importance](https://mlsanity.com/paper/2001.04295/) (2020)
- [Adaptive group-regularized logistic elastic net regression](https://mlsanity.com/paper/1805.00389/) (2018)

## Built on by (most cited first, 9 of 9)

- [Data-Efficient and Interpretable Tabular Anomaly Detection](https://mlsanity.com/paper/2203.02034/) (2022)
- [Interpretable Survival Analysis for Heart Failure Risk Prediction](https://mlsanity.com/paper/2310.15472/) (2023)
- [Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Prediction](https://mlsanity.com/paper/2206.13749/) (2022)
- [SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learning](https://mlsanity.com/paper/2501.15941/) (2025)
- [Interpretable Prediction and Feature Selection for Survival Analysis](https://mlsanity.com/paper/2404.14689/) (2024)
- [ControlBurn: Nonlinear Feature Selection with Sparse Tree Ensembles](https://mlsanity.com/paper/2207.03935/) (2022)
- [FeatEHR-LLM: Leveraging Large Language Models for Feature Engineering in Electronic Health Records](https://mlsanity.com/paper/2604.22534/) (2026)
- [End-to-end Feature Selection Approach for Learning Skinny Trees](https://mlsanity.com/paper/2310.18542/) (2023)
- [FAST: An Optimization Framework for Fast Additive Segmentation in Transparent ML](https://mlsanity.com/paper/2402.12630/) (2024)

## Abstract (arXiv)

Tree ensembles distribute feature importance evenly amongst groups of correlated features. The average feature ranking of the correlated group is suppressed, which reduces interpretability and complicates feature selection. In this paper we present ControlBurn, a feature selection algorithm that uses a weighted LASSO-based feature selection method to prune unnecessary features from tree ensembles, just as low-intensity fire reduces overgrown vegetation. Like the linear LASSO, ControlBurn assigns all the feature importance of a correlated group of features to a single feature. Moreover, the algorithm is efficient and only requires a single training iteration to run, unlike iterative wrapper-based feature selection methods. We show that ControlBurn performs substantially better than feature selection methods with comparable computational costs on datasets with correlated features.