# Width-based Lookaheads with Learnt Base Policies and Heuristics Over the Atari-2600 Benchmark

arXiv: https://arxiv.org/abs/2106.12151 · HTML: https://mlsanity.com/paper/2106.12151/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 26 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 8 of 8)

- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents](https://mlsanity.com/paper/1709.06009/) (2017)
- [State of the Art Control of Atari Games Using Shallow Reinforcement Learning](https://mlsanity.com/paper/1512.01563/) (2015)
- [Planning from Pixels in Atari with Learned Symbolic Representations](https://mlsanity.com/paper/2012.09126/) (2020)
- [Deep Policies for Width-Based Planning in Pixel Domains](https://mlsanity.com/paper/1904.07091/) (2019)
- [Hierarchical Width-Based Planning and Learning](https://mlsanity.com/paper/2101.06177/) (2021)
- [Planning with Pixels in (Almost) Real Time](https://mlsanity.com/paper/1801.03354/) (2018)

## Abstract (arXiv)

We propose new width-based planning and learning algorithms inspired from a careful analysis of the design decisions made by previous width-based planners. The algorithms are applied over the Atari-2600 games and our best performing algorithm, Novelty guided Critical Path Learning (N-CPL), outperforms the previously introduced width-based planning and learning algorithms π-IW(1), π-IW(1)+ and π-HIW(n, 1). Furthermore, we present a taxonomy of the Atari-2600 games according to some of their defining characteristics. This analysis of the games provides further insight into the behaviour and performance of the algorithms introduced. Namely, for games with large branching factors, and games with sparse meaningful rewards, N-CPL outperforms π-IW, π-IW(1)+ and π-HIW(n, 1).