# Interactive Visualization for Debugging RL

arXiv: https://arxiv.org/abs/2008.07331 · HTML: https://mlsanity.com/paper/2008.07331/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2020.

## Built on by (most cited first, 5 of 5)

- [d3rlpy: An Offline Deep Reinforcement Learning Library](https://mlsanity.com/paper/2111.03788/) (2021)
- [Boolean Decision Rules for Reinforcement Learning Policy Summarisation](https://mlsanity.com/paper/2207.08651/) (2022)
- [Toward Debugging Deep Reinforcement Learning Programs with RLExplorer](https://mlsanity.com/paper/2410.04322/) (2024)
- [REVEAL-IT: REinforcement learning with Visibility of Evolving Agent poLicy for InTerpretability](https://mlsanity.com/paper/2406.14214/) (2024)
- [Observer, Not Player: Simulating Theory of Mind in LLMs through Game Observation](https://mlsanity.com/paper/2512.19210/) (2025)

## Abstract (arXiv)

Visualization tools for supervised learning allow users to interpret, introspect, and gain an intuition for the successes and failures of their models. While reinforcement learning practitioners ask many of the same questions, existing tools are not applicable to the RL setting as these tools address challenges typically found in the supervised learning regime. In this work, we design and implement an interactive visualization tool for debugging and interpreting RL algorithms. Our system addresses many features missing from previous tools such as (1) tools for supervised learning often are not interactive; (2) while debugging RL policies researchers use state representations that are different from those seen by the agent; (3) a framework designed to make the debugging RL policies more conducive. We provide an example workflow of how this system could be used, along with ideas for future extensions.