# Neural Random Forests

arXiv: https://arxiv.org/abs/1604.07143 · HTML: https://mlsanity.com/paper/1604.07143/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2016.

## Built on by (most cited first, 25 of 25)

- [ExSpliNet: An interpretable and expressive spline-based neural network](https://mlsanity.com/paper/2205.01510/) (2022)
- [Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent](https://mlsanity.com/paper/2102.07567/) (2021)
- [Feature Inference Attack on Shapley Values](https://mlsanity.com/paper/2407.11359/) (2024)
- [Learning a Decision Tree Algorithm with Transformers](https://mlsanity.com/paper/2402.03774/) (2024)
- [Vanilla Gradient Descent for Oblique Decision Trees](https://mlsanity.com/paper/2408.09135/) (2024)
- [Learning from learning machines: a new generation of AI technology to meet the needs of science](https://mlsanity.com/paper/2111.13786/) (2021)
- [Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guarantees](https://mlsanity.com/paper/2204.07293/) (2022)
- [Towards Adversarially Robust Recommendation from Adaptive Fraudster Detection](https://mlsanity.com/paper/2211.11534/) (2022)
- [Shrub Ensembles for Online Classification](https://mlsanity.com/paper/2112.03723/) (2021)
- [A Hybrid Framework for Sequential Data Prediction with End-to-End Optimization](https://mlsanity.com/paper/2203.13787/) (2022)
- [Sparse tree-based initialization for neural networks](https://mlsanity.com/paper/2209.15283/) (2022)
- [To tree or not to tree? Assessing the impact of smoothing the decision boundaries](https://mlsanity.com/paper/2210.03672/) (2022)
- [Designing ReLU Generative Networks to Enumerate Trees with a Given Tree Edit Distance](https://mlsanity.com/paper/2510.10706/) (2025)
- [Decision Machines: Congruent Decision Trees](https://mlsanity.com/paper/2101.11347/) (2021)
- [Data-driven multinomial random forest: A new random forest variant with strong consistency](https://mlsanity.com/paper/2211.15154/) (2022)
- [Data-driven multinomial random forest](https://mlsanity.com/paper/2304.04240/) (2023)
- [A generalized decision tree ensemble based on the NeuralNetworks architecture: Distributed Gradient Boosting Forest (DGBF)](https://mlsanity.com/paper/2402.03386/) (2024)
- [Conditional Information Gain Trellis](https://mlsanity.com/paper/2402.08345/) (2024)
- [Bayesian Additive Regression Networks](https://mlsanity.com/paper/2404.04425/) (2024)
- [Symbolic Branch Networks: Tree-Inherited Neural Models for Interpretable Multiclass Classification](https://mlsanity.com/paper/2507.01781/) (2025)
- [Decision Tree Embedding by Leaf-Means](https://mlsanity.com/paper/2512.01819/) (2025)
- [Neural Network Conversion of Machine Learning Pipelines](https://mlsanity.com/paper/2603.25699/) (2026)
- [ReLU Networks for Exact Generation of Similar Graphs](https://mlsanity.com/paper/2604.05929/) (2026)
- [Approximation-Free Differentiable Oblique Decision Trees](https://mlsanity.com/paper/2605.07837/) (2026)
- [Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exit](https://mlsanity.com/paper/2609.12337/) (2026)

## Abstract (arXiv)

Given an ensemble of randomized regression trees, it is possible to restructure them as a collection of multilayered neural networks with particular connection weights. Following this principle, we reformulate the random forest method of Breiman (2001) into a neural network setting, and in turn propose two new hybrid procedures that we call neural random forests. Both predictors exploit prior knowledge of regression trees for their architecture, have less parameters to tune than standard networks, and less restrictions on the geometry of the decision boundaries than trees. Consistency results are proved, and substantial numerical evidence is provided on both synthetic and real data sets to assess the excellent performance of our methods in a large variety of prediction problems.