# Oblique Decision Trees from Derivatives of ReLU Networks

arXiv: https://arxiv.org/abs/1909.13488 · HTML: https://mlsanity.com/paper/1909.13488/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2019.

## Built on by (most cited first, 14 of 14)

- [Learning a Decision Tree Algorithm with Transformers](https://mlsanity.com/paper/2402.03774/) (2024)
- [Vanilla Gradient Descent for Oblique Decision Trees](https://mlsanity.com/paper/2408.09135/) (2024)
- [Mixed integer linear optimization formulations for learning optimal binary classification trees](https://mlsanity.com/paper/2206.04857/) (2022)
- [Synthesizing Programmatic Policies with Actor-Critic Algorithms and ReLU Networks](https://mlsanity.com/paper/2308.02729/) (2023)
- [Learning Prescriptive ReLU Networks](https://mlsanity.com/paper/2306.00651/) (2023)
- [NSOTree: Neural Survival Oblique Tree](https://mlsanity.com/paper/2309.13825/) (2023)
- [PEACH: Pretrained-embedding Explanation Across Contextual and Hierarchical Structure](https://mlsanity.com/paper/2404.13645/) (2024)
- [SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks](https://mlsanity.com/paper/2407.14463/) (2024)
- [Lifting Biomolecular Data Acquisition](https://mlsanity.com/paper/2512.15984/) (2025)
- [Decision Machines: Congruent Decision Trees](https://mlsanity.com/paper/2101.11347/) (2021)
- [Multiple Instance Learning with Trainable Decision Tree Ensembles](https://mlsanity.com/paper/2302.06601/) (2023)
- [Equivalent and Compact Representations of Neural Network Controllers With Decision Trees](https://mlsanity.com/paper/2304.06049/) (2023)
- [Unveiling Options with Neural Decomposition](https://mlsanity.com/paper/2410.11262/) (2024)
- [Approximation-Free Differentiable Oblique Decision Trees](https://mlsanity.com/paper/2605.07837/) (2026)

## Abstract (arXiv)

We show how neural models can be used to realize piece-wise constant functions such as decision trees. The proposed architecture, which we call locally constant networks, builds on ReLU networks that are piece-wise linear and hence their associated gradients with respect to the inputs are locally constant. We formally establish the equivalence between the classes of locally constant networks and decision trees. Moreover, we highlight several advantageous properties of locally constant networks, including how they realize decision trees with parameter sharing across branching / leaves. Indeed, only M neurons suffice to implicitly model an oblique decision tree with 2M leaf nodes. The neural representation also enables us to adopt many tools developed for deep networks (e.g., DropConnect (Wan et al., 2013)) while implicitly training decision trees. We demonstrate that our method outperforms alternative techniques for training oblique decision trees in the context of molecular property classification and regression tasks.