# Optimization Methods for Interpretable Differentiable Decision Trees in Reinforcement Learning

arXiv: https://arxiv.org/abs/1903.09338 · HTML: https://mlsanity.com/paper/1903.09338/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2019.

## Built on by (most cited first, 5 of 5)

- [Why do tree-based models still outperform deep learning on tabular data?](https://mlsanity.com/paper/2207.08815/) (2022)
- [Feature-Based Interpretable Reinforcement Learning based on State-Transition Models](https://mlsanity.com/paper/2105.07099/) (2021)
- [Policy Optimization with Sparse Global Contrastive Explanations](https://mlsanity.com/paper/2207.06269/) (2022)
- [Limits of Actor-Critic Algorithms for Decision Tree Policies Learning in IBMDPs](https://mlsanity.com/paper/2309.13365/) (2023)
- [Temporally Interpretable Differentiable Decision Trees](https://mlsanity.com/paper/2610.10367/) (2026)

## Abstract (arXiv)

Decision trees are ubiquitous in machine learning for their ease of use and interpretability. Yet, these models are not typically employed in reinforcement learning as they cannot be updated online via stochastic gradient descent. We overcome this limitation by allowing for a gradient update over the entire tree that improves sample complexity affords interpretable policy extraction. First, we include theoretical motivation on the need for policy-gradient learning by examining the properties of gradient descent over differentiable decision trees. Second, we demonstrate that our approach equals or outperforms a neural network on all domains and can learn discrete decision trees online with average rewards up to 7x higher than a batch-trained decision tree. Third, we conduct a user study to quantify the interpretability of a decision tree, rule list, and a neural network with statistically significant results (p < 0.001).