# Adaptive Forests For Classification

arXiv: https://arxiv.org/abs/2510.22991 · HTML: https://mlsanity.com/paper/2510.22991/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 27 Oct 2025.

## Builds on (most cited first, 4 of 4)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optimal Policy Trees](https://mlsanity.com/paper/2012.02279/) (2020)
- [Optimal Weighted Random Forests](https://mlsanity.com/paper/2305.10042/) (2023)
- [Policy Trees for Prediction: Interpretable and Adaptive Model Selection for Machine Learning](https://mlsanity.com/paper/2405.20486/) (2024)

## Built on by (most cited first, 1 of 1)

- [Decision-Path Patterns as Tree Reliability Signals: Path-based Adaptive Weighting for Random Forest Classification](https://mlsanity.com/paper/2605.20716/) (2026)

## Abstract (arXiv)

Random Forests (RF) and Extreme Gradient Boosting (XGBoost) are two of the most widely used and highly performing classification and regression models. They aggregate equally weighted CART trees, generated randomly in RF or sequentially in XGBoost. In this paper, we propose Adaptive Forests (AF), a novel approach that adaptively selects the weights of the underlying CART models. AF combines (a) the Optimal Predictive-Policy Trees (OP2T) framework to prescribe tailored, input-dependent unequal weights to trees and (b) Mixed Integer Optimization (MIO) to refine weight candidates dynamically, enhancing overall performance. We demonstrate that AF consistently outperforms RF, XGBoost, and other weighted RF in binary and multi-class classification problems over 20+ real-world datasets.