# Simplification of Forest Classifiers and Regressors

arXiv: https://arxiv.org/abs/2212.07103 · HTML: https://mlsanity.com/paper/2212.07103/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 35 references that are in the map.
- First posted on arXiv 14 Dec 2022.

## Builds on (most cited first, 5 of 5)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Machine Learning at the Network Edge: A Survey](https://mlsanity.com/paper/1908.00080/) (2019)
- [Born-Again Tree Ensembles](https://mlsanity.com/paper/2003.11132/) (2020)
- [Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach](https://mlsanity.com/paper/1606.09066/) (2016)
- [Pruning Random Forests for Prediction on a Budget](https://mlsanity.com/paper/1606.05060/) (2016)

## Built on by (most cited first, 1 of 1)

- [Free Lunch in the Forest: Functionally-Identical Pruning of Boosted Tree Ensembles](https://mlsanity.com/paper/2408.16167/) (2024)

## Abstract (arXiv)

We study the problem of sharing as many branching conditions of a given forest classifier or regressor as possible while keeping classification performance. As a constraint for preventing from accuracy degradation, we first consider the one that the decision paths of all the given feature vectors must not change. For a branching condition that a value of a certain feature is at most a given threshold, the set of values satisfying such constraint can be represented as an interval. Thus, the problem is reduced to the problem of finding the minimum set intersecting all the constraint-satisfying intervals for each set of branching conditions on the same feature. We propose an algorithm for the original problem using an algorithm solving this problem efficiently. The constraint is relaxed later to promote further sharing of branching conditions by allowing decision path change of a certain ratio of the given feature vectors or allowing a certain number of non-intersected constraint-satisfying intervals. We also extended our algorithm for both the relaxations. The effectiveness of our method is demonstrated through comprehensive experiments using 21 datasets (13 classification and 8 regression datasets in UCI machine learning repository) and 4 classifiers/regressors (random forest, extremely randomized trees, AdaBoost and gradient boosting).