# Planning from Pixels in Atari with Learned Symbolic Representations

arXiv: https://arxiv.org/abs/2012.09126 · HTML: https://mlsanity.com/paper/2012.09126/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2020.

## Built on by (most cited first, 11 of 11)

- [Learning Neuro-Symbolic Relational Transition Models for Bilevel Planning](https://mlsanity.com/paper/2105.14074/) (2021)
- [Inductive Biases for Object-Centric Representations in the Presence of Complex Textures](https://mlsanity.com/paper/2204.08479/) (2022)
- [The Role of Pretrained Representations for the OOD Generalization of Reinforcement Learning Agents](https://mlsanity.com/paper/2107.05686/) (2021)
- [Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Search](https://mlsanity.com/paper/2212.14849/) (2022)
- [Hierarchical Width-Based Planning and Learning](https://mlsanity.com/paper/2101.06177/) (2021)
- [Neuro-Symbolic Imitation Learning: Discovering Symbolic Abstractions for Skill Learning](https://mlsanity.com/paper/2503.21406/) (2025)
- [Is Policy Learning Overrated?: Width-Based Planning and Active Learning for Atari](https://mlsanity.com/paper/2109.15310/) (2021)
- [Discrete World Models via Regularization](https://mlsanity.com/paper/2603.01748/) (2026)
- [Width-based Lookaheads with Learnt Base Policies and Heuristics Over the Atari-2600 Benchmark](https://mlsanity.com/paper/2106.12151/) (2021)
- [Symbolic Distillation for Learned TCP Congestion Control](https://mlsanity.com/paper/2210.16987/) (2022)
- [See, Symbolize, Act: Grounding VLMs with Spatial Representations for Better Gameplay](https://mlsanity.com/paper/2603.11601/) (2026)

## Abstract (arXiv)

Width-based planning methods have been shown to yield state-of-the-art performance in the Atari 2600 domain using pixel input. One successful approach, RolloutIW, represents states with the B-PROST boolean feature set. An augmented version of RolloutIW, π-IW, shows that learned features can be competitive with handcrafted ones for width-based search. In this paper, we leverage variational autoencoders (VAEs) to learn features directly from pixels in a principled manner, and without supervision. The inference model of the trained VAEs extracts boolean features from pixels, and RolloutIW plans with these features. The resulting combination outperforms the original RolloutIW and human professional play on Atari 2600 and drastically reduces the size of the feature set.