# Decision Forests, Convolutional Networks and the Models in-Between

arXiv: https://arxiv.org/abs/1603.01250 · HTML: https://mlsanity.com/paper/1603.01250/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2016.

## Built on by (most cited first, 15 of 15)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning](https://mlsanity.com/paper/2106.03760/) (2021)
- [Shapley Explanation Networks](https://mlsanity.com/paper/2104.02297/) (2021)
- [Fast Feedforward Networks](https://mlsanity.com/paper/2308.14711/) (2023)
- [A Survey of Neural Trees](https://mlsanity.com/paper/2209.03415/) (2022)
- [Tree-gated Deep Mixture-of-Experts For Pose-robust Face Alignment](https://mlsanity.com/paper/1910.09450/) (2019)
- [Memorization Capacity of Neural Networks with Conditional Computation](https://mlsanity.com/paper/2303.11247/) (2023)
- [COMET: Learning Cardinality Constrained Mixture of Experts with Trees and Local Search](https://mlsanity.com/paper/2306.02824/) (2023)
- [Online Feature Screening for Data Streams with Concept Drift](https://mlsanity.com/paper/2104.02883/) (2021)
- [Switchable Representation Learning Framework with Self-compatibility](https://mlsanity.com/paper/2206.08289/) (2022)
- [DecisioNet: A Binary-Tree Structured Neural Network](https://mlsanity.com/paper/2207.01127/) (2022)
- [Dive into Decision Trees and Forests: A Theoretical Demonstration](https://mlsanity.com/paper/2101.08656/) (2021)
- [Making CNNs Interpretable by Building Dynamic Sequential Decision Forests with Top-down Hierarchy Learning](https://mlsanity.com/paper/2106.02824/) (2021)
- [Deep Convolutional Tables: Deep Learning without Convolutions](https://mlsanity.com/paper/2304.11706/) (2023)
- [Conditional Information Gain Trellis](https://mlsanity.com/paper/2402.08345/) (2024)

## Abstract (arXiv)

This paper investigates the connections between two state of the art classifiers: decision forests (DFs, including decision jungles) and convolutional neural networks (CNNs). Decision forests are computationally efficient thanks to their conditional computation property (computation is confined to only a small region of the tree, the nodes along a single branch). CNNs achieve state of the art accuracy, thanks to their representation learning capabilities. We present a systematic analysis of how to fuse conditional computation with representation learning and achieve a continuum of hybrid models with different ratios of accuracy vs. efficiency. We call this new family of hybrid models conditional networks. Conditional networks can be thought of as: i) decision trees augmented with data transformation operators, or ii) CNNs, with block-diagonal sparse weight matrices, and explicit data routing functions. Experimental validation is performed on the common task of image classification on both the CIFAR and Imagenet datasets. Compared to state of the art CNNs, our hybrid models yield the same accuracy with a fraction of the compute cost and much smaller number of parameters.