mlsanityData to 7 Oct 2026

A Survey of Neural Trees

arXiv 2209.03415 ↗First posted 7 Sep 2022cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Neural networks (NNs) and decision trees (DTs) are both popular models of machine learning, yet coming with mutually exclusive advantages and limitations. To bring the best of the two worlds, a variety of approaches are proposed to integrate NNs and DTs explicitly or implicitly. In this survey, these approaches are organized in a school which we term as neural trees (NTs). This survey aims to present a comprehensive review of NTs and attempts to identify how they enhance the model interpretability. We first propose a thorough taxonomy of NTs that expresses the gradual integration and co-evolution of NNs and DTs. Afterward, we analyze NTs in terms of their interpretability and performance, and suggest possible solutions to the remaining challenges. Finally, this survey concludes with a discussion about other considerations like conditional computation and promising directions towards this field. A list of papers reviewed in this survey, along with their corresponding codes, is available at: https://github.com/zju-vipa/awesome-neural-trees

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 60 of its 177 references that are in the map. First posted on arXiv 7 Sep 2022.

Builds on

60 of 177 references.

References · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  11. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  12. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
38 more
  1. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  2. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  3. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  4. 2018Techniques for Interpretable Machine Learningcited by 237
  5. 2017Distilling a Neural Network Into a Soft Decision Treecited by 230
  6. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 213
  7. 2014Evaluation of Output Embeddings for Fine-Grained Image Classificationcited by 166
  8. 2015Conditional Computation in Neural Networks for faster modelscited by 166
  9. 2018Visual Interpretability for Deep Learning: a Surveycited by 155
  10. 2016Latent Embeddings for Zero-shot Classificationcited by 129
  11. 2019Making Better Mistakes: Leveraging Class Hierarchies with Deep Networkscited by 112
  12. 2018Interpreting CNNs via Decision Treescited by 89
  13. 2020Explaining the Explainer: A First Theoretical Analysis of LIMEcited by 89
  14. 2020On Interpretability of Artificial Neural Networks: A Surveycited by 88
  15. 2013Deep Learning of Representations: Looking Forwardcited by 84
  16. 2018Deep Neural Decision Treescited by 72
  17. 2017Beyond Sparsity: Tree Regularization of Deep Models for Interpretabilitycited by 68
  18. 2020Coherent Hierarchical Multi-Label Classification Networkscited by 65
  19. 2018Adaptive Neural Treescited by 64
  20. 2017Do Convolutional Neural Networks Learn Class Hierarchy?cited by 61
  21. 2016Network of Experts for Large-Scale Image Categorizationcited by 54
  22. 2019Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorizationcited by 51
  23. 2018Interpretable Convolutional Neural Networks via Feedforward Designcited by 49
  24. 2020NBDT: Neural-Backed Decision Treescited by 46
  25. 2020Hierarchical Image Classification using Entailment Cone Embeddingscited by 46
  26. 2018Hierarchy-based Image Embeddings for Semantic Image Retrievalcited by 41
  27. 2017Deep Regression Forests for Age Estimationcited by 35
  28. 2017Deciding How to Decide: Dynamic Routing in Artificial Neural Networkscited by 33
  29. 2018Explainable Neural Networks based on Additive Index Modelscited by 33
  30. 2018Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solutioncited by 31
  31. 2022Label Relation Graphs Enhanced Hierarchical Residual Network for Hierarchical Multi-Granularity Classificationcited by 29
  32. 2018Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embeddingcited by 28
  33. 2019Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretationcited by 22
  34. 2018Deep Supervision with Intermediate Conceptscited by 19
  35. 2017Neural Decision Treescited by 17
  36. 2018Global Model Interpretation via Recursive Partitioningcited by 17
  37. 2016Decision Forests, Convolutional Networks and the Models in-Betweencited by 15
  38. 2017Multi-Scale Dense Networks for Resource Efficient Image Classificationcited by 15

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 1; 2026: 2.

7 papers.

  1. 20233VL: Using Trees to Improve Vision-Language Models' Interpretabilitycited by 7
  2. 2025From GNNs to Trees: Multi-Granular Interpretability for Graph Neural Networkscited by 4
  3. 2024Efficient Normalized Conformal Prediction and Uncertainty Quantification for Anti-Cancer Drug Sensitivity Prediction with Deep Regression Forestscited by 1
  4. 2024Forecasting with Hyper-Treescited by 1
  5. 2024Vehicle Behavior Prediction by Episodic-Memory Implanted NDTcited by 0
  6. 2026Enhancing LIME using Neural Decision Treescited by 0
  7. 2026MotherTree: Meta-learning on synthetic data improves decision tree trainingcited by 0