mlsanityData to 7 Oct 2026

GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent

arXiv 2305.03515 ↗First posted 5 May 2023cs.LG cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Decision Trees (DTs) are commonly used for many machine learning tasks due to their high degree of interpretability. However, learning a DT from data is a difficult optimization problem, as it is non-convex and non-differentiable. Therefore, common approaches learn DTs using a greedy growth algorithm that minimizes the impurity locally at each internal node. Unfortunately, this greedy procedure can lead to inaccurate trees. In this paper, we present a novel approach for learning hard, axis-aligned DTs with gradient descent. The proposed method uses backpropagation with a straight-through operator on a dense DT representation, to jointly optimize all tree parameters. Our approach outperforms existing methods on binary classification benchmarks and achieves competitive results for multi-class tasks. The method is available under: https://github.com/s-marton/GradTree

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 19 of its 39 references that are in the map. First posted on arXiv 5 May 2023.

Builds on

19 of 39 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  3. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  4. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  5. 2017Distilling a Neural Network Into a Soft Decision Treecited by 230
  6. 2019Neural Oblivious Decision Ensembles for Deep Learning on Tabular Datacited by 200
  7. 2019Sparse Sequence-to-Sequence Modelscited by 167
  8. 2020Generalized and Scalable Optimal Sparse Decision Treescited by 98
  9. 2021NODE-GAM: Neural Generalized Additive Model for Interpretable Deep Learningcited by 88
  10. 2018Deep Neural Decision Treescited by 72
  11. 2018Adaptive Neural Treescited by 64
  12. 2022PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functionscited by 62
7 more
  1. 2020MurTree: Optimal Classification Trees via Dynamic Programming and Searchcited by 61
  2. 2022Quant-BnB: A Scalable Branch-and-Bound Method for Optimal Decision Trees with Continuous Featurescited by 19
  3. 2015Efficient non-greedy optimization of decision treescited by 17
  4. 2021Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descentcited by 14
  5. 2020Sparsity in Optimal Randomized Classification Treescited by 13
  6. 2020Learning Binary Decision Trees by Argmin Differentiationcited by 11
  7. 2020Learning Representations for Axis-Aligned Decision Forests through Input Perturbationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 6; 2026: 2.

11 papers.

  1. 2023GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Datacited by 10
  2. 2024Optimal or Greedy Decision Trees? Revisiting their Objectives, Tuning, and Performancecited by 7
  3. 2025Decision Tree Induction Through LLMs via Semantically-Aware Evolutioncited by 6
  4. 2024Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimizationcited by 5
  5. 2025Hierarchical Retrieval at Scale: Bridging Transparency and Efficiencycited by 1
  6. 2025MOSIC: Model-Agnostic Optimal Subgroup Identification with Multi-Constraint for Improved Reliabilitycited by 1
  7. 2025Decision Trees That Remember: Gradient-Based Learning of Recurrent Decision Trees with Memorycited by 0
  8. 2025Neurosymbolic Decision Treescited by 0
  9. 2025LHT: Statistically-Driven Oblique Decision Trees for Interpretable Classificationcited by 0
  10. 2026Approximation-Free Differentiable Oblique Decision Treescited by 0
  11. 2026Adaptive Multi-Branching for Shallow Decision Tree Inductioncited by 0