# Enhance Learning Efficiency of Oblique Decision Tree via Feature Concatenation

arXiv: https://arxiv.org/abs/2502.00465 · HTML: https://mlsanity.com/paper/2502.00465/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 42 references that are in the map.
- First posted on arXiv 1 Feb 2025.

## Builds on (most cited first, 10 of 10)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [A Scalable MIP-based Method for Learning Optimal Multivariate Decision Trees](https://mlsanity.com/paper/2011.03375/) (2020)
- [Sparse Projection Oblique Randomer Forests](https://mlsanity.com/paper/1506.03410/) (2015)
- [Multi-Label Learning with Deep Forest](https://mlsanity.com/paper/1911.06557/) (2019)
- [Convergence Rates of Oblique Regression Trees for Flexible Function Libraries](https://mlsanity.com/paper/2210.14429/) (2022)
- [Quality Diversity Evolutionary Learning of Decision Trees](https://mlsanity.com/paper/2208.12758/) (2022)
- [Oblique Predictive Clustering Trees](https://mlsanity.com/paper/2007.13617/) (2020)
- [Analyzing the tree-layer structure of Deep Forests](https://mlsanity.com/paper/2010.15690/) (2020)
- [Oblique and rotation double random forest](https://mlsanity.com/paper/2111.02010/) (2021)

## Abstract (arXiv)

Oblique Decision Tree (ODT) separates the feature space by linear projections, as opposed to the conventional Decision Tree (DT) that forces axis-parallel splits. ODT has been proven to have a stronger representation ability than DT, as it provides a way to create shallower tree structures while still approximating complex decision boundaries. However, its learning efficiency is still insufficient, since the linear projections cannot be transmitted to the child nodes, resulting in a waste of model parameters. In this work, we propose an enhanced ODT method with Feature Concatenation (textttFC-ODT), which enables in-model feature transformation to transmit the projections along the decision paths. Theoretically, we prove that our method enjoys a faster consistency rate w.r.t. the tree depth, indicating that our method possesses a significant advantage in generalization performance, especially for shallow trees. Experiments show that textttFC-ODT can outperform the other state-of-the-art decision trees with a limited tree depth.