# Investigating Simple Object Representations in Model-Free Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2002.06703 · HTML: https://mlsanity.com/paper/2002.06703/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2020.

## Built on by (most cited first, 5 of 5)

- [A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2106.02097/) (2021)
- [Structured World Belief for Reinforcement Learning in POMDP](https://mlsanity.com/paper/2107.08577/) (2021)
- [Deep Reinforcement Learning via Object-Centric Attention](https://mlsanity.com/paper/2504.03024/) (2025)
- [Diagnosing and exploiting the computational demands of videos games for deep reinforcement learning](https://mlsanity.com/paper/2309.13181/) (2023)
- [Boosting Deep Reinforcement Learning with Semantic Knowledge for Robotic Manipulators](https://mlsanity.com/paper/2601.16866/) (2026)

## Abstract (arXiv)

We explore the benefits of augmenting state-of-the-art model-free deep reinforcement algorithms with simple object representations. Following the Frostbite challenge posited by Lake et al. (2017), we identify object representations as a critical cognitive capacity lacking from current reinforcement learning agents. We discover that providing the Rainbow model (Hessel et al.,2018) with simple, feature-engineered object representations substantially boosts its performance on the Frostbite game from Atari 2600. We then analyze the relative contributions of the representations of different types of objects, identify environment states where these representations are most impactful, and examine how these representations aid in generalizing to novel situations.