# NBDT: Neural-Backed Decision Trees

arXiv: https://arxiv.org/abs/2004.00221 · HTML: https://mlsanity.com/paper/2004.00221/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2020.

## Built on by (most cited first, 30 of 46)

- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [Leveraging Sparse Linear Layers for Debuggable Deep Networks](https://mlsanity.com/paper/2105.04857/) (2021)
- [Fine-Grained Scene Graph Generation with Data Transfer](https://mlsanity.com/paper/2203.11654/) (2022)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [Behavioral Intention Prediction in Driving Scenes: A Survey](https://mlsanity.com/paper/2211.00385/) (2022)
- [Privacy-preserving Federated Adversarial Domain Adaption over Feature Groups for Interpretability](https://mlsanity.com/paper/2111.10934/) (2021)
- [Proto2Proto: Can you recognize the car, the way I do?](https://mlsanity.com/paper/2204.11830/) (2022)
- [Neural Networks are Decision Trees](https://mlsanity.com/paper/2210.05189/) (2022)
- [Attributable Visual Similarity Learning](https://mlsanity.com/paper/2203.14932/) (2022)
- [LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies](https://mlsanity.com/paper/2407.16067/) (2024)
- [VIPriors 1: Visual Inductive Priors for Data-Efficient Deep Learning Challenges](https://mlsanity.com/paper/2103.03768/) (2021)
- [Coarse-to-Fine Curriculum Learning](https://mlsanity.com/paper/2106.04072/) (2021)
- [SketchXAI: A First Look at Explainability for Human Sketches](https://mlsanity.com/paper/2304.11744/) (2023)
- [A Survey of Neural Trees](https://mlsanity.com/paper/2209.03415/) (2022)
- [SEAL: Simultaneous Label Hierarchy Exploration And Learning](https://mlsanity.com/paper/2304.13374/) (2023)
- [Vanilla Gradient Descent for Oblique Decision Trees](https://mlsanity.com/paper/2408.09135/) (2024)
- [Bort: Towards Explainable Neural Networks with Bounded Orthogonal Constraint](https://mlsanity.com/paper/2212.09062/) (2022)
- [Path Choice Matters for Clear Attribution in Path Methods](https://mlsanity.com/paper/2401.10442/) (2024)
- [Taxonomic Networks: A Representation for Neuro-Symbolic Pairing](https://mlsanity.com/paper/2505.24601/) (2025)
- [Finding Representative Interpretations on Convolutional Neural Networks](https://mlsanity.com/paper/2108.06384/) (2021)
- [Efficient Fraud Detection Using Deep Boosting Decision Trees](https://mlsanity.com/paper/2302.05918/) (2023)
- [Contrastive Hierarchical Clustering](https://mlsanity.com/paper/2303.03389/) (2023)
- [Language Model as Visual Explainer](https://mlsanity.com/paper/2412.07802/) (2024)
- [MetaDT: Meta Decision Tree with Class Hierarchy for Interpretable Few-Shot Learning](https://mlsanity.com/paper/2203.01482/) (2022)
- [HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm](https://mlsanity.com/paper/2605.24797/) (2026)
- [SONG: Self-Organizing Neural Graphs](https://mlsanity.com/paper/2107.13214/) (2021)
- [Transition Matrix Representation of Trees with Transposed Convolutions](https://mlsanity.com/paper/2202.10677/) (2022)
- [Fair Feature Importance Scores for Interpreting Tree-Based Methods and Surrogates](https://mlsanity.com/paper/2310.04352/) (2023)
- [Foundations of Multisensory Artificial Intelligence](https://mlsanity.com/paper/2404.18976/) (2024)
- [Is Data Valuation Learnable and Interpretable?](https://mlsanity.com/paper/2406.02612/) (2024)

## Abstract (arXiv)

Machine learning applications such as finance and medicine demand accurate and justifiable predictions, barring most deep learning methods from use. In response, previous work combines decision trees with deep learning, yielding models that (1) sacrifice interpretability for accuracy or (2) sacrifice accuracy for interpretability. We forgo this dilemma by jointly improving accuracy and interpretability using Neural-Backed Decision Trees (NBDTs). NBDTs replace a neural network's final linear layer with a differentiable sequence of decisions and a surrogate loss. This forces the model to learn high-level concepts and lessens reliance on highly-uncertain decisions, yielding (1) accuracy: NBDTs match or outperform modern neural networks on CIFAR, ImageNet and better generalize to unseen classes by up to 16%. Furthermore, our surrogate loss improves the original model's accuracy by up to 2%. NBDTs also afford (2) interpretability: improving human trustby clearly identifying model mistakes and assisting in dataset debugging. Code and pretrained NBDTs are at https://github.com/alvinwan/neural-backed-decision-trees.