# Modelling Agent Policies with Interpretable Imitation Learning

arXiv: https://arxiv.org/abs/2006.11309 · HTML: https://mlsanity.com/paper/2006.11309/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 2 of its 8 references that are in the map.
- First posted on arXiv 19 Jun 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 2 of 2)

- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [State Representation Learning for Control: An Overview](https://mlsanity.com/paper/1802.04181/) (2018)

## Built on by (most cited first, 6 of 6)

- [Inverse Contextual Bandits: Learning How Behavior Evolves over Time](https://mlsanity.com/paper/2107.06317/) (2021)
- [The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation](https://mlsanity.com/paper/2106.04240/) (2021)
- [Inverse Decision Modeling: Learning Interpretable Representations of Behavior](https://mlsanity.com/paper/2310.18591/) (2023)
- [Explaining Imitation Learning through Frames](https://mlsanity.com/paper/2301.01088/) (2023)
- [Symbolic Imitation Learning: From Black-Box to Explainable Driving Policies](https://mlsanity.com/paper/2309.16025/) (2023)
- [Interpretable Robotic Manipulation from Language](https://mlsanity.com/paper/2405.17047/) (2024)

## Abstract (arXiv)

As we deploy autonomous agents in safety-critical domains, it becomes important to develop an understanding of their internal mechanisms and representations. We outline an approach to imitation learning for reverse-engineering black box agent policies in MDP environments, yielding simplified, interpretable models in the form of decision trees. As part of this process, we explicitly model and learn agents' latent state representations by selecting from a large space of candidate features constructed from the Markov state. We present initial promising results from an implementation in a multi-agent traffic environment.