# Fast Exploration with Simplified Models and Approximately Optimistic Planning in Model Based Reinforcement Learning

arXiv: https://arxiv.org/abs/1806.00175 · HTML: https://mlsanity.com/paper/1806.00175/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2018.

## Built on by (most cited first, 5 of 5)

- [Object-Centric Learning with Slot Mixture Module](https://mlsanity.com/paper/2311.04640/) (2023)
- [Relational Object-Centric Actor-Critic](https://mlsanity.com/paper/2310.17178/) (2023)
- [Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction](https://mlsanity.com/paper/2408.11816/) (2024)
- [Symbolic Disentangled Representations for Images](https://mlsanity.com/paper/2412.19847/) (2024)
- [Relate to Predict: Towards Task-Independent Knowledge Representations for Reinforcement Learning](https://mlsanity.com/paper/2212.05298/) (2022)

## Abstract (arXiv)

Humans learn to play video games significantly faster than the state-of-the-art reinforcement learning (RL) algorithms. People seem to build simple models that are easy to learn to support planning and strategic exploration. Inspired by this, we investigate two issues in leveraging model-based RL for sample efficiency. First we investigate how to perform strategic exploration when exact planning is not feasible and empirically show that optimistic Monte Carlo Tree Search outperforms posterior sampling methods. Second we show how to learn simple deterministic models to support fast learning using object representation. We illustrate the benefit of these ideas by introducing a novel algorithm, Strategic Object Oriented Reinforcement Learning (SOORL), that outperforms state-of-the-art algorithms in the game of Pitfall! in less than 50 episodes.