# An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents

arXiv: https://arxiv.org/abs/1812.07069 · HTML: https://mlsanity.com/paper/1812.07069/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2018.

## Built on by (most cited first, 15 of 15)

- [Contrastive Learning as Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2206.07568/) (2022)
- [Model Zoos: A Dataset of Diverse Populations of Neural Network Models](https://mlsanity.com/paper/2209.14764/) (2022)
- [Investigating the Properties of Neural Network Representations in Reinforcement Learning](https://mlsanity.com/paper/2203.15955/) (2022)
- [Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data](https://mlsanity.com/paper/2306.03346/) (2023)
- [Bayesian Nonparametrics for Offline Skill Discovery](https://mlsanity.com/paper/2202.04675/) (2022)
- [Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework](https://mlsanity.com/paper/2411.11761/) (2024)
- [Human-AI Shared Control via Policy Dissection](https://mlsanity.com/paper/2206.00152/) (2022)
- [An Unsupervised Video Game Playstyle Metric via State Discretization](https://mlsanity.com/paper/2110.00950/) (2021)
- [Deep Reinforcement Learning with Vector Quantized Encoding](https://mlsanity.com/paper/2211.06733/) (2022)
- [Abstracted Trajectory Visualization for Explainability in Reinforcement Learning](https://mlsanity.com/paper/2402.07928/) (2024)
- [Deep Innovation Protection: Confronting the Credit Assignment Problem in Training Heterogeneous Neural Architectures](https://mlsanity.com/paper/2001.01683/) (2019)
- [The Sandbox Environment for Generalizable Agent Research (SEGAR)](https://mlsanity.com/paper/2203.10351/) (2022)
- [RAMario: Experimental Approach to Reptile Algorithm -- Reinforcement Learning for Mario](https://mlsanity.com/paper/2305.09655/) (2023)
- [Unsupervised Salient Patch Selection for Data-Efficient Reinforcement Learning](https://mlsanity.com/paper/2402.03329/) (2024)
- [A Differentiable Atari VCS:A Complex, Fully Known Ground Truth for Explainable AI](https://mlsanity.com/paper/2606.22447/) (2026)

## Abstract (arXiv)

Much human and computational effort has aimed to improve how deep reinforcement learning algorithms perform on benchmarks such as the Atari Learning Environment. Comparatively less effort has focused on understanding what has been learned by such methods, and investigating and comparing the representations learned by different families of reinforcement learning (RL) algorithms. Sources of friction include the onerous computational requirements, and general logistical and architectural complications for running Deep RL algorithms at scale. We lessen this friction, by (1) training several algorithms at scale and releasing trained models, (2) integrating with a previous Deep RL model release, and (3) releasing code that makes it easy for anyone to load, visualize, and analyze such models. This paper introduces the Atari Zoo framework, which contains models trained across benchmark Atari games, in an easy-to-use format, as well as code that implements common modes of analysis and connects such models to a popular neural network visualization library. Further, to demonstrate the potential of this dataset and software package, we show initial quantitative and qualitative comparisons between the performance and representations of several deep RL algorithms, highlighting interesting and previously unknown distinctions between them.