# Atari-5: Distilling the Arcade Learning Environment down to Five Games

arXiv: https://arxiv.org/abs/2210.02019 · HTML: https://mlsanity.com/paper/2210.02019/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 5 Oct 2022.

## Builds on (most cited first, 18 of 18)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Unifying Count-Based Exploration and Intrinsic Motivation](https://mlsanity.com/paper/1606.01868/) (2016)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Leveraging Procedural Generation to Benchmark Reinforcement Learning](https://mlsanity.com/paper/1912.01588/) (2019)
- [Implicit Quantile Networks for Distributional Reinforcement Learning](https://mlsanity.com/paper/1806.06923/) (2018)
- [Agent57: Outperforming the Atari Human Benchmark](https://mlsanity.com/paper/2003.13350/) (2020)
- [Distributed Prioritized Experience Replay](https://mlsanity.com/paper/1803.00933/) (2018)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning](https://mlsanity.com/paper/1605.02097/) (2016)
- [Playing hard exploration games by watching YouTube](https://mlsanity.com/paper/1805.11592/) (2018)
- [Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Research](https://mlsanity.com/paper/2011.14826/) (2020)
- [Gotta Learn Fast: A New Benchmark for Generalization in RL](https://mlsanity.com/paper/1804.03720/) (2018)
- [Compress and Control](https://mlsanity.com/paper/1411.5326/) (2014)
- [Conjugated Discrete Distributions for Distributional Reinforcement Learning](https://mlsanity.com/paper/2112.07424/) (2021)

## Built on by (most cited first, 30 of 44)

- [Simplifying Deep Temporal Difference Learning](https://mlsanity.com/paper/2407.04811/) (2024)
- [LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrations](https://mlsanity.com/paper/2412.01441/) (2024)
- [OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environments](https://mlsanity.com/paper/2306.08649/) (2023)
- [No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO](https://mlsanity.com/paper/2405.00662/) (2024)
- [Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning](https://mlsanity.com/paper/2506.15544/) (2025)
- [Thinker: Learning to Plan and Act](https://mlsanity.com/paper/2307.14993/) (2023)
- [The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networks](https://mlsanity.com/paper/2506.03404/) (2025)
- [Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agents](https://mlsanity.com/paper/2510.13704/) (2025)
- [Scaling Laws for Imitation Learning in Single-Agent Games](https://mlsanity.com/paper/2307.09423/) (2023)
- [HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learning](https://mlsanity.com/paper/2406.03997/) (2024)
- [Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps](https://mlsanity.com/paper/2412.17113/) (2024)
- [SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores](https://mlsanity.com/paper/2306.16688/) (2023)
- [Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC](https://mlsanity.com/paper/2411.03820/) (2024)
- [Hadamax Encoding: Elevating Performance in Model-Free Atari](https://mlsanity.com/paper/2505.15345/) (2025)
- [Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities](https://mlsanity.com/paper/2507.13158/) (2025)
- [Stable Deep Reinforcement Learning via Isotropic Gaussian Representations](https://mlsanity.com/paper/2602.19373/) (2026)
- [Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning](https://mlsanity.com/paper/2405.16195/) (2024)
- [ARLBench: Flexible and Efficient Benchmarking for Hyperparameter Optimization in Reinforcement Learning](https://mlsanity.com/paper/2409.18827/) (2024)
- [Reinforcement Learning From Imperfect Corrective Actions And Proxy Rewards](https://mlsanity.com/paper/2410.05782/) (2024)
- [Reliability-Adjusted Prioritized Experience Replay](https://mlsanity.com/paper/2506.18482/) (2025)
- [The Formalism-Implementation Gap in Reinforcement Learning Research](https://mlsanity.com/paper/2510.16175/) (2025)
- [Understanding and Improving Hyperbolic Deep Reinforcement Learning](https://mlsanity.com/paper/2512.14202/) (2025)
- [Performance Variation in Deep Reinforcement Learning](https://mlsanity.com/paper/2606.06746/) (2026)
- [Distributional Reinforcement Learning with Dual Expectile-Quantile Regression](https://mlsanity.com/paper/2305.16877/) (2023)
- [Experience Replay with Random Reshuffling](https://mlsanity.com/paper/2503.02269/) (2025)
- [ULTHO: Ultra-Lightweight yet Efficient Hyperparameter Optimization in Deep Reinforcement Learning](https://mlsanity.com/paper/2503.06101/) (2025)
- [Deep Double Q-learning](https://mlsanity.com/paper/2507.00275/) (2025)
- [Best Agent Identification for General Game Playing](https://mlsanity.com/paper/2507.00451/) (2025)
- [Is Exploration or Optimization the Problem for Deep Reinforcement Learning?](https://mlsanity.com/paper/2508.01329/) (2025)
- [Stackelberg Coupling of Online Representation Learning and Reinforcement Learning](https://mlsanity.com/paper/2508.07452/) (2025)

## Abstract (arXiv)

The Arcade Learning Environment (ALE) has become an essential benchmark for assessing the performance of reinforcement learning algorithms. However, the computational cost of generating results on the entire 57-game dataset limits ALE's use and makes the reproducibility of many results infeasible. We propose a novel solution to this problem in the form of a principled methodology for selecting small but representative subsets of environments within a benchmark suite. We applied our method to identify a subset of five ALE games, called Atari-5, which produces 57-game median score estimates within 10% of their true values. Extending the subset to 10-games recovers 80% of the variance for log-scores for all games within the 57-game set. We show this level of compression is possible due to a high degree of correlation between many of the games in ALE.