# Interpretable Reinforcement Learning for Robotics and Continuous Control

arXiv: https://arxiv.org/abs/2311.10041 · HTML: https://mlsanity.com/paper/2311.10041/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2023.

## Built on by (most cited first, 5 of 5)

- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [Synthesizing Interpretable Control Policies through Large Language Model Guided Search](https://mlsanity.com/paper/2410.05406/) (2024)
- [SMOSE: Sparse Mixture of Shallow Experts for Interpretable Reinforcement Learning in Continuous Control Tasks](https://mlsanity.com/paper/2412.13053/) (2024)
- [Towards Automated Semantic Interpretability in Reinforcement Learning via Vision-Language Models](https://mlsanity.com/paper/2503.16724/) (2025)
- [Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving](https://mlsanity.com/paper/2505.06737/) (2025)

## Abstract (arXiv)

Interpretability in machine learning is critical for the safe deployment of learned policies across legally-regulated and safety-critical domains. While gradient-based approaches in reinforcement learning have achieved tremendous success in learning policies for continuous control problems such as robotics and autonomous driving, the lack of interpretability is a fundamental barrier to adoption. We propose Interpretable Continuous Control Trees (ICCTs), a tree-based model that can be optimized via modern, gradient-based, reinforcement learning approaches to produce high-performing, interpretable policies. The key to our approach is a procedure for allowing direct optimization in a sparse decision-tree-like representation. We validate ICCTs against baselines across six domains, showing that ICCTs are capable of learning policies that parity or outperform baselines by up to 33% in autonomous driving scenarios while achieving a 300x-600x reduction in the number of parameters against deep learning baselines. We prove that ICCTs can serve as universal function approximators and display analytically that ICCTs can be verified in linear time. Furthermore, we deploy ICCTs in two realistic driving domains, based on interstate Highway-94 and 280 in the US. Finally, we verify ICCT's utility with end-users and find that ICCTs are rated easier to simulate, quicker to validate, and more interpretable than neural networks.