# Free Lunch in the Forest: Functionally-Identical Pruning of Boosted Tree Ensembles

arXiv: https://arxiv.org/abs/2408.16167 · HTML: https://mlsanity.com/paper/2408.16167/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 28 Aug 2024.

## Builds on (most cited first, 12 of 12)

- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks](https://mlsanity.com/paper/2203.04466/) (2022)
- [Fast as CHITA: Neural Network Pruning with Combinatorial Optimization](https://mlsanity.com/paper/2302.14623/) (2023)
- [Born-Again Tree Ensembles](https://mlsanity.com/paper/2003.11132/) (2020)
- [Optimal Counterfactual Explanations in Tree Ensembles](https://mlsanity.com/paper/2106.06631/) (2021)
- [Lossless Compression of Deep Neural Networks](https://mlsanity.com/paper/2001.00218/) (2020)
- [To tune or not to tune the number of trees in random forest?](https://mlsanity.com/paper/1705.05654/) (2017)
- [OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization](https://mlsanity.com/paper/2403.12983/) (2024)
- [ForestPrune: Compact Depth-Controlled Tree Ensembles](https://mlsanity.com/paper/2206.00128/) (2022)
- [Improving the Accuracy-Memory Trade-Off of Random Forests Via Leaf-Refinement](https://mlsanity.com/paper/2110.10075/) (2021)
- [Simplification of Forest Classifiers and Regressors](https://mlsanity.com/paper/2212.07103/) (2022)

## Built on by (most cited first, 4 of 4)

- [Boosting Revisited: Benchmarking and Advancing LP-Based Ensemble Methods](https://mlsanity.com/paper/2507.18242/) (2025)
- [Boosted Trees on a Diet: Compact Models for Resource-Constrained Devices](https://mlsanity.com/paper/2510.26557/) (2025)
- [PACE: Prune-And-Compress Ensemble Models](https://mlsanity.com/paper/2605.06278/) (2026)
- [PINE: Pruning Boosted Tree Ensembles with Conformal In-Distribution Prediction Equivalence](https://mlsanity.com/paper/2605.28068/) (2026)

## Abstract (arXiv)

Tree ensembles, including boosting methods, are highly effective and widely used for tabular data. However, large ensembles lack interpretability and require longer inference times. We introduce a method to prune a tree ensemble into a reduced version that is "functionally identical" to the original model. In other words, our method guarantees that the prediction function stays unchanged for any possible input. As a consequence, this pruning algorithm is lossless for any aggregated metric. We formalize the problem of functionally identical pruning on ensembles, introduce an exact optimization model, and provide a fast yet highly effective method to prune large ensembles. Our algorithm iteratively prunes considering a finite set of points, which is incrementally augmented using an adversarial model. In multiple computational experiments, we show that our approach is a "free lunch", significantly reducing the ensemble size without altering the model's behavior. Thus, we can preserve state-of-the-art performance at a fraction of the original model's size.