# SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks

arXiv: https://arxiv.org/abs/2411.12173 · HTML: https://mlsanity.com/paper/2411.12173/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 19 Nov 2024.

## Placed in

- Deep reinforcement learning methods / [Interpretable and logic-guided RL](https://mlsanity.com/deeprl/interpretable-and-logic-guided-rl/)

## Builds on (most cited first, 28 of 28)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks](https://mlsanity.com/paper/2112.03227/) (2021)
- [Learning Latent Plans from Play](https://mlsanity.com/paper/1903.01973/) (2019)
- [Distilling a Neural Network Into a Soft Decision Tree](https://mlsanity.com/paper/1711.09784/) (2017)
- [Accelerating Reinforcement Learning with Learned Skill Priors](https://mlsanity.com/paper/2010.11944/) (2020)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [Explainable Reinforcement Learning Through a Causal Lens](https://mlsanity.com/paper/1905.10958/) (2019)
- [Visualizing and Understanding Atari Agents](https://mlsanity.com/paper/1711.00138/) (2017)
- [Explainability in Deep Reinforcement Learning](https://mlsanity.com/paper/2008.06693/) (2020)
- [SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning](https://mlsanity.com/paper/1811.00090/) (2018)
- [Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning](https://mlsanity.com/paper/1712.07294/) (2017)
- [Demonstration-Guided Reinforcement Learning with Learned Skills](https://mlsanity.com/paper/2107.10253/) (2021)
- [Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees](https://mlsanity.com/paper/1807.05887/) (2018)
- [Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives](https://mlsanity.com/paper/2110.15360/) (2021)
- [TACO: Learning Task Decomposition via Temporal Alignment for Control](https://mlsanity.com/paper/1803.01840/) (2018)
- [Learning Robot Skills with Temporal Variational Inference](https://mlsanity.com/paper/2006.16232/) (2020)
- [Interpretable Policies for Reinforcement Learning by Genetic Programming](https://mlsanity.com/paper/1712.04170/) (2017)
- [Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning](https://mlsanity.com/paper/2101.12446/) (2021)
- [Explaining Reinforcement Learning to Mere Mortals: An Empirical Study](https://mlsanity.com/paper/1903.09708/) (2019)
- [TripleTree: A Versatile Interpretable Representation of Black Box Agents and their Environments](https://mlsanity.com/paper/2009.04743/) (2020)
- [CDT: Cascading Decision Trees for Explainable Reinforcement Learning](https://mlsanity.com/paper/2011.07553/) (2020)
- [Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning](https://mlsanity.com/paper/2010.10903/) (2020)
- [Mo"ET: Mixture of Expert Trees and its Application to Verifiable Reinforcement Learning](https://mlsanity.com/paper/1906.06717/) (2019)
- [Learning Symbolic Rules for Interpretable Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08228/) (2021)
- [Interpretable Rule Discovery Through Bilevel Optimization of Split-Rules of Nonlinear Decision Trees for Classification Problems](https://mlsanity.com/paper/2008.00410/) (2020)
- [Synthesizing Programmatic Policies with Actor-Critic Algorithms and ReLU Networks](https://mlsanity.com/paper/2308.02729/) (2023)

## Built on by (most cited first, 3 of 3)

- [E2DT: Efficient and Effective Decision Transformer with Experience-Aware Sampling for Robotic Manipulation](https://mlsanity.com/paper/2605.00159/) (2026)
- [ConceptTree: Bringing Semantic Transparency to Black-Box Decision Making for Robotic Manipulation](https://mlsanity.com/paper/2607.17861/) (2026)
- [Safe Task Planning with Long-Term Graph Memory for Embodied Agents](https://mlsanity.com/paper/2609.08444/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (DRL) has achieved remarkable success in various research domains. However, its reliance on neural networks results in a lack of transparency, which limits its practical applications. To achieve explainability, decision trees have emerged as a popular and promising alternative to neural networks. Nonetheless, due to their limited expressiveness, traditional decision trees struggle with high-dimensional long-horizon continuous control tasks. In this paper, we proposes SkillTree, a novel framework that reduces complex continuous action spaces into discrete skill spaces. Our hierarchical approach integrates a differentiable decision tree within the high-level policy to generate skill embeddings, which subsequently guide the low-level policy in executing skills. By making skill decisions explainable, we achieve skill-level explainability, enhancing the understanding of the decision-making process in complex tasks. Experimental results demonstrate that our method achieves performance comparable to skill-based neural networks in complex robotic arm control domains. Furthermore, SkillTree offers explanations at the skill level, thereby increasing the transparency of the decision-making process.