# Neural Decision Trees

arXiv: https://arxiv.org/abs/1702.07360 · HTML: https://mlsanity.com/paper/1702.07360/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2017.

## Built on by (most cited first, 17 of 17)

- [Mechanistic Mode Connectivity](https://mlsanity.com/paper/2211.08422/) (2022)
- [ExSpliNet: An interpretable and expressive spline-based neural network](https://mlsanity.com/paper/2205.01510/) (2022)
- [Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent](https://mlsanity.com/paper/2102.07567/) (2021)
- [A Survey of Neural Trees](https://mlsanity.com/paper/2209.03415/) (2022)
- [Trade-off Between Efficiency and Consistency for Removal-based Explanations](https://mlsanity.com/paper/2210.17426/) (2022)
- [Mixed integer linear optimization formulations for learning optimal binary classification trees](https://mlsanity.com/paper/2206.04857/) (2022)
- [FUTURE: Flexible Unlearning for Tree Ensemble](https://mlsanity.com/paper/2508.21181/) (2025)
- [Intra-video Positive Pairs in Self-Supervised Learning for Ultrasound](https://mlsanity.com/paper/2403.07715/) (2024)
- [Prophecy: Inferring Formal Properties from Neuron Activations](https://mlsanity.com/paper/2509.21677/) (2025)
- [To tree or not to tree? Assessing the impact of smoothing the decision boundaries](https://mlsanity.com/paper/2210.03672/) (2022)
- [Dive into Decision Trees and Forests: A Theoretical Demonstration](https://mlsanity.com/paper/2101.08656/) (2021)
- [Approach to Learning Generalized Audio Representation Through Batch Embedding Covariance Regularization and Constant-Q Transforms](https://mlsanity.com/paper/2303.03591/) (2023)
- [LEURN: Learning Explainable Univariate Rules with Neural Networks](https://mlsanity.com/paper/2303.14937/) (2023)
- [Optimal Mixed Integer Linear Optimization Trained Multivariate Classification Trees](https://mlsanity.com/paper/2408.01297/) (2024)
- [Triadic Concept Analysis for Logic Interpretation of Simple Artificial Networks](https://mlsanity.com/paper/2601.06229/) (2026)
- [Enhancing LIME using Neural Decision Trees](https://mlsanity.com/paper/2603.20919/) (2026)
- [Discovery of fully efficient fault indicators along a data-based diagnosis process](https://mlsanity.com/paper/2609.28087/) (2026)

## Abstract (arXiv)

In this paper we propose a synergistic melting of neural networks and decision trees (DT) we call neural decision trees (NDT). NDT is an architecture a la decision tree where each splitting node is an independent multilayer perceptron allowing oblique decision functions or arbritrary nonlinear decision function if more than one layer is used. This way, each MLP can be seen as a node of the tree. We then show that with the weight sharing asumption among those units, we end up with a Hashing Neural Network (HNN) which is a multilayer perceptron with sigmoid activation function for the last layer as opposed to the standard softmax. The output units then jointly represent the probability to be in a particular region. The proposed framework allows for global optimization as opposed to greedy in DT and differentiability w.r.t. all parameters and the input, allowing easy integration in any learnable pipeline, for example after CNNs for computer vision tasks. We also demonstrate the modeling power of HNN allowing to learn union of disjoint regions for final clustering or classification making it more general and powerful than standard softmax MLP requiring linear separability thus reducing the need on the inner layer to perform complex data transformations. We finally show experiments for supervised, semi-suppervised and unsupervised tasks and compare results with standard DTs and MLPs.