# Centroid Decision Forest

arXiv: https://arxiv.org/abs/2503.19306 · HTML: https://mlsanity.com/paper/2503.19306/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 52 references that are in the map.
- First posted on arXiv 25 Mar 2025.

## Builds on (most cited first, 12 of 12)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Predicting Many Properties of a Quantum System from Very Few Measurements](https://mlsanity.com/paper/2002.08953/) (2020)
- [Consistency of random forests](https://mlsanity.com/paper/1405.2881/) (2014)
- [Random-projection ensemble classification](https://mlsanity.com/paper/1504.04595/) (2015)
- [Minimax optimal rates for Mondrian trees and forests](https://mlsanity.com/paper/1803.05784/) (2018)
- [Depth Selection for Deep ReLU Nets in Feature Extraction and Generalization](https://mlsanity.com/paper/2004.00245/) (2020)
- [Estimating the Algorithmic Variance of Randomized Ensembles via the Bootstrap](https://mlsanity.com/paper/1907.08742/) (2019)
- [Realization of spatial sparseness by deep ReLU nets with massive data](https://mlsanity.com/paper/1912.07464/) (2019)
- [Innovated interaction screening for high-dimensional nonlinear classification](https://mlsanity.com/paper/1501.01029/) (2015)
- [Feature Selection via Robust Weighted Score for High Dimensional Binary Class-Imbalanced Gene Expression Data](https://mlsanity.com/paper/2401.12667/) (2024)

## Abstract (arXiv)

This paper introduces the centroid decision forest (CDF), a novel ensemble learning framework that redefines the splitting strategy and tree building in the ordinary decision trees for high-dimensional classification. The splitting approach in CDF differs from the traditional decision trees in theat the class separability score (CSS) determines the selection of the most discriminative features at each node to construct centroids of the partitions (daughter nodes). The splitting criterion uses the Euclidean distance measurements from each class centroid to achieve a splitting mechanism that is more flexible and robust. Centroids are constructed by computing the mean feature values of the selected features for each class, ensuring a class-representative division of the feature space. This centroid-driven approach enables CDF to capture complex class structures while maintaining interpretability and scalability. To evaluate CDF, 23 high-dimensional datasets are used to assess its performance against different state-of-the-art classifiers through classification accuracy and Cohen's kappa statistic. The experimental results show that CDF outperforms the conventional methods establishing its effectiveness and flexibility for high-dimensional classification problems.