# Interpreting Neural Policies with Disentangled Tree Representations

arXiv: https://arxiv.org/abs/2210.06650 · HTML: https://mlsanity.com/paper/2210.06650/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 54 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 30 of 32)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/1710.00935/) (2017)
- [Liquid Time-constant Networks](https://mlsanity.com/paper/2006.04439/) (2020)
- [Liquid Structural State-Space Models](https://mlsanity.com/paper/2209.12951/) (2022)
- [Learning Deep Disentangled Embeddings with the F-Statistic Loss](https://mlsanity.com/paper/1802.05312/) (2018)
- [Interpreting CNNs via Decision Trees](https://mlsanity.com/paper/1802.00121/) (2018)
- [Learning Long-Term Dependencies in Irregularly-Sampled Time Series](https://mlsanity.com/paper/2006.04418/) (2020)
- [VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles](https://mlsanity.com/paper/2111.12083/) (2021)
- [Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy](https://mlsanity.com/paper/2103.03014/) (2021)
- [Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability](https://mlsanity.com/paper/1912.01752/) (2019)
- [Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing](https://mlsanity.com/paper/2103.04909/) (2021)
- [On The Verification of Neural ODEs with Stochastic Guarantees](https://mlsanity.com/paper/2012.08863/) (2020)
- [Causal Navigation by Continuous-time Neural Networks](https://mlsanity.com/paper/2106.08314/) (2021)
- [Adversarial Training is Not Ready for Robot Learning](https://mlsanity.com/paper/2103.08187/) (2021)
- [On-Off Center-Surround Receptive Fields for Accurate and Robust Image Classification](https://mlsanity.com/paper/2106.07091/) (2021)
- [Optimizing for Interpretability in Deep Neural Networks with Tree Regularization](https://mlsanity.com/paper/1908.05254/) (2019)
- [Sparse Flows: Pruning Continuous-depth Models](https://mlsanity.com/paper/2106.12718/) (2021)
- [BarrierNet: A Safety-Guaranteed Layer for Neural Networks](https://mlsanity.com/paper/2111.11277/) (2021)
- [On the Forward Invariance of Neural ODEs](https://mlsanity.com/paper/2210.04763/) (2022)
- [Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks](https://mlsanity.com/paper/1809.03864/) (2018)
- [Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Driving](https://mlsanity.com/paper/2203.02401/) (2022)
- [Enhancing Decision Tree based Interpretation of Deep Neural Networks through L1-Orthogonal Regularization](https://mlsanity.com/paper/1904.05394/) (2019)
- [Extracting Interpretable Concept-Based Decision Trees from CNNs](https://mlsanity.com/paper/1906.04664/) (2019)
- [Interactive Analysis of CNN Robustness](https://mlsanity.com/paper/2110.07667/) (2021)

## Built on by (most cited first, 1 of 1)

- [Efficient Dataset Distillation Using Random Feature Approximation](https://mlsanity.com/paper/2210.12067/) (2022)

## Abstract (arXiv)

The advancement of robots, particularly those functioning in complex human-centric environments, relies on control solutions that are driven by machine learning. Understanding how learning-based controllers make decisions is crucial since robots are often safety-critical systems. This urges a formal and quantitative understanding of the explanatory factors in the interpretability of robot learning. In this paper, we aim to study interpretability of compact neural policies through the lens of disentangled representation. We leverage decision trees to obtain factors of variation [1] for disentanglement in robot learning; these encapsulate skills, behaviors, or strategies toward solving tasks. To assess how well networks uncover the underlying task dynamics, we introduce interpretability metrics that measure disentanglement of learned neural dynamics from a concentration of decisions, mutual information and modularity perspective. We showcase the effectiveness of the connection between interpretability and disentanglement consistently across extensive experimental analysis.