# Statistical Advantages of Oblique Randomized Decision Trees and Forests

arXiv: https://arxiv.org/abs/2407.02458 · HTML: https://mlsanity.com/paper/2407.02458/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 43 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 17 of 17)

- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Consistency of random forests](https://mlsanity.com/paper/1405.2881/) (2014)
- [Iterative Random Forests to detect predictive and stable high-order interactions](https://mlsanity.com/paper/1706.08457/) (2017)
- [Large Scale Prediction with Decision Trees](https://mlsanity.com/paper/2104.13881/) (2021)
- [Minimax optimal rates for Mondrian trees and forests](https://mlsanity.com/paper/1803.05784/) (2018)
- [Sparse Projection Oblique Randomer Forests](https://mlsanity.com/paper/1506.03410/) (2015)
- [Estimation and Inference with Trees and Forests in High Dimensions](https://mlsanity.com/paper/2007.03210/) (2020)
- [Canonical Correlation Forests](https://mlsanity.com/paper/1507.05444/) (2015)
- [Analysis of purely random forests bias](https://mlsanity.com/paper/1407.3939/) (2014)
- [Convergence Rates of Oblique Regression Trees for Flexible Function Libraries](https://mlsanity.com/paper/2210.14429/) (2022)
- [Random Tessellation Forests](https://mlsanity.com/paper/1906.05440/) (2019)
- [Stochastic geometry to generalize the Mondrian Process](https://mlsanity.com/paper/2002.00797/) (2020)
- [Inference with Mondrian Random Forests](https://mlsanity.com/paper/2310.09702/) (2023)
- [The Binary Space Partitioning-Tree Process](https://mlsanity.com/paper/1903.09343/) (2019)
- [Minimax Rates for High-Dimensional Random Tessellation Forests](https://mlsanity.com/paper/2109.10541/) (2021)
- [Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guarantees](https://mlsanity.com/paper/2204.07293/) (2022)
- [Dimension Reduction Forests: Local Variable Importance using Structured Random Forests](https://mlsanity.com/paper/2103.13233/) (2021)

## Built on by (most cited first, 1 of 1)

- [TrIM: Transformed Iterative Mondrian Forests for Gradient-based Dimension Reduction and High-Dimensional Regression](https://mlsanity.com/paper/2407.09964/) (2024)

## Abstract (arXiv)

This work studies the statistical implications of using features comprised of general linear combinations of covariates to partition the data in randomized decision tree and forest regression algorithms. Using random tessellation theory in stochastic geometry, we provide a theoretical analysis of a class of efficiently generated random tree and forest estimators that allow for oblique splits along such features. We call these estimators oblique Mondrian trees and forests, as the trees are generated by first selecting a set of features from linear combinations of the covariates and then running a Mondrian process that hierarchically partitions the data along these features. Quadratic risk bounds and convergence rates are obtained for the flexible function class of multi-index models for dimension reduction, where the output is assumed to depend on a low-dimensional relevant feature subspace of the input domain. The results highlight how the risk of these estimators depends on the choice of features and quantify how robust the risk is with respect to error between the selected features along which the data is split and the true relevant feature subspace. The asymptotic analysis also provides conditions on the convergence rate a set of estimated relevant features must satisfy for oblique Mondrian estimators to obtain minimax optimal rates of convergence with respect to the dimension of the relevant feature subspace. Additionally, a lower bound on the risk of axis-aligned Mondrian trees (where features are restricted to the set of covariates) is obtained, proving that these estimators are suboptimal for general ridge functions, no matter how the distribution over the covariates used to divide the data at each tree node is weighted.