mlsanityData to 14 Oct 2026

OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environments

arXiv 2306.08649 ↗First posted 14 Jun 2023cs.LG cs.AI cs.CVcited by 30

Not on the map. Why?

Abstract (arXiv)

Cognitive science and psychology suggest that object-centric representations of complex scenes are a promising step towards enabling efficient abstract reasoning from low-level perceptual features. Yet, most deep reinforcement learning approaches only rely on pixel-based representations that do not capture the compositional properties of natural scenes. For this, we need environments and datasets that allow us to work and evaluate object-centric approaches. In our work, we extend the Atari Learning Environments, the most-used evaluation framework for deep RL approaches, by introducing OCAtari, that performs resource-efficient extractions of the object-centric states for these games. Our framework allows for object discovery, object representation learning, as well as object-centric RL. We evaluate OCAtari's detection capabilities and resource efficiency. Our source code is available at github.com/k4ntz/OC_Atari.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 56 of its 77 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

56 of 77 references.

References · 56

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2020Mastering Atari with Discrete World Modelscited by 1,007
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  8. 2020Object-Centric Learning with Slot Attentioncited by 876
  9. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  10. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  11. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  12. 2017A Distributional Perspective on Reinforcement Learningcited by 765
38 more
  1. 2017Deep Reinforcement Learning that Matterscited by 754
  2. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  3. 2019Model-Based Reinforcement Learning for Ataricited by 502
  4. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  5. 2018VirtualHome: Simulating Household Activities via Programscited by 415
  6. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  7. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
  8. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  9. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  10. 2020First return, then explorecited by 266
  11. 2021Conditional Object-Centric Learning from Videocited by 248
  12. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  13. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  14. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  15. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  16. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 194
  17. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  18. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 166
  19. 2021Illiterate DALL-E Learns to Composecited by 147
  20. 2019Unsupervised State Representation Learning in Ataricited by 144
  21. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
  22. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  23. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  24. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 123
  25. 2018Generalization and Regularization in DQNcited by 118
  26. 2021Generalization and Robustness Implications in Object-Centric Learningcited by 81
  27. 2019Neural Logic Reinforcement Learningcited by 53
  28. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  29. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 46
  30. 2023Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manualscited by 45
  31. 2022Atari-5: Distilling the Arcade Learning Environment down to Five Gamescited by 44
  32. 2018Investigating Human Priors for Playing Video Gamescited by 39
  33. 2022Human-level Atari 200x fastercited by 28
  34. 2021Adaptive Rational Activations to Boost Deep Reinforcement Learningcited by 23
  35. 2022Compositional Multi-Object Reinforcement Learning with Linear Relation Networkscited by 19
  36. 2021Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learningcited by 18
  37. 2022Boosting Object Representation Learning via Motion and Object Continuitycited by 17
  38. 2019Is Deep Reinforcement Learning Really Superhuman on Atari? Leveling the playing fieldcited by 15

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 10; 2026: 9.

30 papers.

Sort:

By area:

  1. 2024Gymnasium: A Standard Interface for Reinforcement Learning Environmentscited by 46
  2. 2025PoE-World: Compositional World Modeling with Products of Programmatic Expertscited by 26
  3. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  4. 2024Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedbackcited by 21
  5. 2024Pix2Code: Learning to Compose Neural Visual Concepts as Programscited by 18
  6. 2024Interpretable and Editable Programmatic Tree Policies for Reinforcement Learningcited by 18
  7. 2024End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanationscited by 17
  8. 2024BlendRL: A Framework for Merging Symbolic and Neural Policy Learningcited by 14
  9. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  10. 2025Object-Centric World Models from Few-Shot Annotations for Sample-Efficient Reinforcement Learningcited by 10
  11. 2025One Life to Learn: Inferring Symbolic World Models for Stochastic Environments from Unguided Explorationcited by 10
  12. 2025Learning Game-Playing Agents with Generative Code Optimizationcited by 8
18 more
  1. 2026Understanding the Challenges in Iterative Generative Optimization with LLMscited by 8
  2. 2026Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusioncited by 7
  3. 2024EXPIL: Explanatory Predicate Invention for Learning in Gamescited by 5
  4. 2024LICORICE: Label-Efficient Concept-Based Interpretable Reinforcement Learningcited by 5
  5. 2024Interpretable end-to-end Neurosymbolic Reinforcement Learning agentscited by 4
  6. 2025Evaluating Interpretable Reinforcement Learning by Distilling Policies into Programscited by 3
  7. 2025Better Decisions through the Right Causal World Modelcited by 3
  8. 2025The Formalism-Implementation Gap in Reinforcement Learning Researchcited by 3
  9. 2025Deep Reinforcement Learning via Object-Centric Attentioncited by 2
  10. 2025DyMoDreamer: World Modeling with Dynamic Modulationcited by 2
  11. 2025Human-Allied Relational Reinforcement Learningcited by 1
  12. 2026Joint Learning of Hierarchical Neural Options and Abstract World Modelcited by 1
  13. 2026Boosting deep Reinforcement Learning using pretraining with Logical Optionscited by 0
  14. 2026See, Symbolize, Act: Grounding VLMs with Spatial Representations for Better Gameplaycited by 0
  15. 2026GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learningcited by 0
  16. 2026Language Game: Talking to Non-Human Systemscited by 0
  17. 2026Mind-Studio: Executable World Models with Lookahead Evaluation for Partially Observable Gamescited by 0
  18. 2026A Differentiable Atari VCS:A Complex, Fully Known Ground Truth for Explainable AIcited by 0