mlsanityData to 7 Oct 2026

Atari-5: Distilling the Arcade Learning Environment down to Five Games

arXiv 2210.02019 ↗First posted 5 Oct 2022cs.AI cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

The Arcade Learning Environment (ALE) has become an essential benchmark for assessing the performance of reinforcement learning algorithms. However, the computational cost of generating results on the entire 57-game dataset limits ALE's use and makes the reproducibility of many results infeasible. We propose a novel solution to this problem in the form of a principled methodology for selecting small but representative subsets of environments within a benchmark suite. We applied our method to identify a subset of five ALE games, called Atari-5, which produces 57-game median score estimates within 10% of their true values. Extending the subset to 10-games recovers 80% of the variance for log-scores for all games within the 57-game set. We show this level of compression is possible due to a high degree of correlation between many of the games in ALE.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 18 of its 34 references that are in the map. First posted on arXiv 5 Oct 2022.

Builds on

18 of 34 references.

References · 18

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  5. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  6. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  7. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  8. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  9. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  10. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  11. 2018Distributed Prioritized Experience Replaycited by 260
  12. 2020First return, then explorecited by 256
6 more
  1. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 178
  2. 2018Playing hard exploration games by watching YouTubecited by 98
  3. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  4. 2018Gotta Learn Fast: A New Benchmark for Generalization in RLcited by 59
  5. 2014Compress and Controlcited by 7
  6. 2021Conjugated Discrete Distributions for Distributional Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 9; 2025: 19; 2026: 11.

44 papers.

Sort:

By area:

  1. 2024Simplifying Deep Temporal Difference Learningcited by 71
  2. 2024LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrationscited by 32
  3. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  4. 2024No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPOcited by 27
  5. 2025Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learningcited by 22
  6. 2023Thinker: Learning to Plan and Actcited by 13
  7. 2025The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networkscited by 12
  8. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  9. 2023Scaling Laws for Imitation Learning in Single-Agent Gamescited by 11
  10. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  11. 2024Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timestepscited by 11
  12. 2023SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Corescited by 10
32 more
  1. 2024Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PCcited by 7
  2. 2025Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunitiescited by 7
  3. 2025Hadamax Encoding: Elevating Performance in Model-Free Ataricited by 6
  4. 2026Stable Deep Reinforcement Learning via Isotropic Gaussian Representationscited by 5
  5. 2024Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learningcited by 3
  6. 2024ARLBench: Flexible and Efficient Benchmarking for Hyperparameter Optimization in Reinforcement Learningcited by 3
  7. 2025Reliability-Adjusted Prioritized Experience Replaycited by 3
  8. 2025The Formalism-Implementation Gap in Reinforcement Learning Researchcited by 3
  9. 2024Reinforcement Learning From Imperfect Corrective Actions And Proxy Rewardscited by 2
  10. 2026Performance Variation in Deep Reinforcement Learningcited by 2
  11. 2023Distributional Reinforcement Learning with Dual Expectile-Quantile Regressioncited by 1
  12. 2025Experience Replay with Random Reshufflingcited by 1
  13. 2025ULTHO: Ultra-Lightweight yet Efficient Hyperparameter Optimization in Deep Reinforcement Learningcited by 1
  14. 2025Deep Double Q-learningcited by 1
  15. 2025Best Agent Identification for General Game Playingcited by 1
  16. 2025Is Exploration or Optimization the Problem for Deep Reinforcement Learning?cited by 1
  17. 2025Stackelberg Coupling of Online Representation Learning and Reinforcement Learningcited by 1
  18. 2025Understanding and Improving Hyperbolic Deep Reinforcement Learningcited by 1
  19. 2026Direct Advantage Estimation for Scalable and Sample-efficient Deep Reinforcement Learningcited by 1
  20. 2025On the Mistaken Assumption of Interchangeable Deep Reinforcement Learning Implementationscited by 0
  21. 2025ANO : Faster is Better in Noisy Landscapecited by 0
  22. 2025Efficient Reinforcement Learning by Reducing Forgetting with Elephant Activation Functionscited by 0
  23. 2025ES-C51: Expected Sarsa Based C51 Distributional Reinforcement Learning Algorithmcited by 0
  24. 2025Recurrent Off-Policy Deep Reinforcement Learning Doesn't Have to be Slowcited by 0
  25. 2026Target-Aligned Reinforcement Learningcited by 0
  26. 2026Approximate Next Policy Sampling: Replacing Conservative Target Policy Updates in Deep RLcited by 0
  27. 2026Revisiting Adam for Streaming Reinforcement Learningcited by 0
  28. 2026Quantile Geometry Regularization for Distributional Reinforcement Learningcited by 0
  29. 2026Scalable Reinforcement Learning via Adaptive Batch Scalingcited by 0
  30. 2026Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learningcited by 0
  31. 2026Accelerating Q-learning through Efficient Value-Sharing across Actionscited by 0
  32. 2026Endpoint Replay: Compressing the Recency Buffer in Deep Reinforcement Learningcited by 0