mlsanityData to 7 Oct 2026

Distilling a Neural Network Into a Soft Decision Tree

arXiv 1711.09784 ↗First posted 27 Nov 2017cs.LG cs.AI stat.MLcited by 230

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have proved to be a very effective way to perform classification tasks. They excel when the input data is high dimensional, the relationship between the input and the output is complicated, and the number of labeled training examples is large. But it is hard to explain why a learned network makes a particular classification decision on a particular test case. This is due to their reliance on distributed hierarchical representations. If we could take the knowledge acquired by the neural net and express the same knowledge in a model that relies on hierarchical decisions instead, explaining a particular decision would be much easier. We describe a way of using a trained neural net to create a type of soft decision tree that generalizes better than one learned directly from the training data.

Key facts. Cited by 230 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 52; 2022: 43; 2023: 50; 2024: 42; 2025: 24; 2026: 19.

230 papers.

Sort:

By area:

  1. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  2. 2021Dynamic Neural Networks: A Surveycited by 405
  3. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  4. 2021Trustworthy AI: From Principles to Practicescited by 206
  5. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  6. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  7. 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 93
  8. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  9. 2021Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyondcited by 75
  10. 2021Improving Deep Learning Interpretability by Saliency Guided Trainingcited by 68
  11. 2021A Survey on Ensemble Learning under the Era of Deep Learningcited by 47
  12. 2022Logic-Based Explainability in Machine Learningcited by 47
38 more
  1. 2024Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitatecited by 46
  2. 2021Scalable Rule-Based Representation Learning for Interpretable Classificationcited by 40
  3. 2021STEEX: Steering Counterfactual Explanations with Semanticscited by 40
  4. 2023Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discoverycited by 39
  5. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  6. 2022Augmenting Interpretable Models with LLMs during Trainingcited by 36
  7. 2022Cycle-Consistent Counterfactuals by Latent Transformationscited by 33
  8. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  9. 2021Adaptive wavelet distillation from neural networks through interpretationscited by 32
  10. 2022OCTET: Object-aware Counterfactual Explanationscited by 28
  11. 2023Does a Neural Network Really Encode Symbolic Concepts?cited by 27
  12. 2022Interpretable by Design: Learning Predictors by Composing Interpretable Queriescited by 26
  13. 2023A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?cited by 24
  14. 2022Computing Abductive Explanations for Boosted Treescited by 23
  15. 2021BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networkscited by 17
  16. 2022Interpretable Mixture of Expertscited by 17
  17. 2022Neural Networks are Decision Treescited by 16
  18. 2021Keep CALM and Improve Visual Feature Attributioncited by 15
  19. 2023Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approachcited by 15
  20. 2021Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descentcited by 14
  21. 2021Explainable Machine Learning with Prior Knowledge: An Overviewcited by 14
  22. 2022POETREE: Interpretable Policy Learning with Adaptive Decision Treescited by 14
  23. 2023Learning Interpretable Rules for Scalable Data Representation and Classificationcited by 14
  24. 2024Towards a theory of model distillationcited by 14
  25. 2021Latent Space Explanation by Interventioncited by 13
  26. 2022Reinforced Path Reasoning for Counterfactual Explainable Recommendationcited by 13
  27. 2024Towards the Dynamics of a DNN Learning Symbolic Interactionscited by 13
  28. 2022Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunitiescited by 12
  29. 2022Learning Interpretable, High-Performing Policies for Autonomous Drivingcited by 11
  30. 2023GradTree: Learning Axis-Aligned Decision Trees with Gradient Descentcited by 11
  31. 2021Label-Descriptive Patterns and Their Application to Characterizing Classification Errorscited by 10
  32. 2022MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Treescited by 10
  33. 2023Tree Variational Autoencoderscited by 10
  34. 2023GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Datacited by 10
  35. 2021Interpretable Machine Learning: Moving From Mythos to Diagnosticscited by 9
  36. 2022Superpolynomial Lower Bounds for Decision Tree Learning and Testingcited by 9
  37. 2024Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspectivecited by 9
  38. 2022ProGReST: Prototypical Graph Regression Soft Trees for Molecular Property Predictioncited by 8

50 most cited of 230 All 230 (JSON)