# Distilling Reinforcement Learning Policies for Interpretable Robot Locomotion: Gradient Boosting Machines and Symbolic Regression

arXiv: https://arxiv.org/abs/2403.14328 · HTML: https://mlsanity.com/paper/2403.14328/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [Multi-expert learning of adaptive legged locomotion](https://mlsanity.com/paper/2012.05810/) (2020)
- [Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl](https://mlsanity.com/paper/2305.01582/) (2023)
- [Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation](https://mlsanity.com/paper/1904.06703/) (2019)
- [Learning Visual Locomotion with Cross-Modal Supervision](https://mlsanity.com/paper/2211.03785/) (2022)
- [TripleTree: A Versatile Interpretable Representation of Black Box Agents and their Environments](https://mlsanity.com/paper/2009.04743/) (2020)
- [Explainable Autonomous Robots: A Survey and Perspective](https://mlsanity.com/paper/2105.02658/) (2021)
- [Hierarchical generative modelling for autonomous robots](https://mlsanity.com/paper/2308.07775/) (2023)
- [Learning Quadruped Locomotion Policies using Logical Rules](https://mlsanity.com/paper/2107.10969/) (2021)
- [Identifying Important Sensory Feedback for Learning Locomotion Skills](https://mlsanity.com/paper/2306.17101/) (2023)
- [Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations](https://mlsanity.com/paper/2109.14026/) (2021)
- [XAI-N: Sensor-based Robot Navigation using Expert Policies and Decision Trees](https://mlsanity.com/paper/2104.10818/) (2021)
- [Learning Vision-Guided Dynamic Locomotion Over Challenging Terrains](https://mlsanity.com/paper/2109.04322/) (2021)
- [Modular Neural Network Policies for Learning In-Flight Object Catching with a Robot Hand-Arm System](https://mlsanity.com/paper/2312.13987/) (2023)

## Built on by (most cited first, 5 of 5)

- [Evaluating Interpretable Reinforcement Learning by Distilling Policies into Programs](https://mlsanity.com/paper/2503.08322/) (2025)
- [Uncovering Latent Phase Structures and Branching Logic in Locomotion Policies: A Case Study on HalfCheetah](https://mlsanity.com/paper/2603.18084/) (2026)
- [SCDP: Learning Humanoid Locomotion from Partial Observations via Mixed-Observation Distillation](https://mlsanity.com/paper/2603.09574/) (2026)
- [Evidence of an Emergent "Self" in Continual Robot Learning](https://mlsanity.com/paper/2603.24350/) (2026)
- [Recovering Hidden Reward in Diffusion-Based Policies](https://mlsanity.com/paper/2605.00623/) (2026)

## Abstract (arXiv)

Recent advancements in reinforcement learning (RL) have led to remarkable achievements in robot locomotion capabilities. However, the complexity and ``black-box'' nature of neural network-based RL policies hinder their interpretability and broader acceptance, particularly in applications demanding high levels of safety and reliability. This paper introduces a novel approach to distill neural RL policies into more interpretable forms using Gradient Boosting Machines (GBMs), Explainable Boosting Machines (EBMs) and Symbolic Regression. By leveraging the inherent interpretability of generalized additive models, decision trees, and analytical expressions, we transform opaque neural network policies into more transparent ``glass-box'' models. We train expert neural network policies using RL and subsequently distill them into (i) GBMs, (ii) EBMs, and (iii) symbolic policies. To address the inherent distribution shift challenge of behavioral cloning, we propose to use the Dataset Aggregation (DAgger) algorithm with a curriculum of episode-dependent alternation of actions between expert and distilled policies, to enable efficient distillation of feedback control policies. We evaluate our approach on various robot locomotion gaits -- walking, trotting, bounding, and pacing -- and study the importance of different observations in joint actions for distilled policies using various methods. We train neural expert policies for 205 hours of simulated experience and distill interpretable policies with only 10 minutes of simulated interaction for each gait using the proposed method.