# Planning with Pixels in (Almost) Real Time

arXiv: https://arxiv.org/abs/1801.03354 · HTML: https://mlsanity.com/paper/1801.03354/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2018.

## Built on by (most cited first, 5 of 5)

- [Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Search](https://mlsanity.com/paper/2212.14849/) (2022)
- [Hierarchical Width-Based Planning and Learning](https://mlsanity.com/paper/2101.06177/) (2021)
- [Planning for Novelty: Width-Based Algorithms for Common Problems in Control, Planning and Reinforcement Learning](https://mlsanity.com/paper/2106.04866/) (2021)
- [Is Policy Learning Overrated?: Width-Based Planning and Active Learning for Atari](https://mlsanity.com/paper/2109.15310/) (2021)
- [Width-based Lookaheads with Learnt Base Policies and Heuristics Over the Atari-2600 Benchmark](https://mlsanity.com/paper/2106.12151/) (2021)

## Abstract (arXiv)

Recently, width-based planning methods have been shown to yield state-of-the-art results in the Atari 2600 video games. For this, the states were associated with the (RAM) memory states of the simulator. In this work, we consider the same planning problem but using the screen instead. By using the same visual inputs, the planning results can be compared with those of humans and learning methods. We show that the planning approach, out of the box and without training, results in scores that compare well with those obtained by humans and learning methods, and moreover, by developing an episodic, rollout version of the IW(k) algorithm, we show that such scores can be obtained in almost real time.