# Practical Open-Loop Optimistic Planning

arXiv: https://arxiv.org/abs/1904.04700 · HTML: https://mlsanity.com/paper/1904.04700/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2019.

## Built on by (most cited first, 4 of 4)

- [Addressing Action Oscillations through Learning Policy Inertia](https://mlsanity.com/paper/2103.02287/) (2021)
- [Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge](https://mlsanity.com/paper/2103.10245/) (2021)
- [Planning in entropy-regularized Markov decision processes and games](https://mlsanity.com/paper/2604.19695/) (2026)
- [Scale-free adaptive planning for deterministic dynamics & discounted rewards](https://mlsanity.com/paper/2604.18312/) (2026)

## Abstract (arXiv)

We consider the problem of online planning in a Markov Decision Process when given only access to a generative model, restricted to open-loop policies - i.e. sequences of actions - and under budget constraint. In this setting, the Open-Loop Optimistic Planning (OLOP) algorithm enjoys good theoretical guarantees but is overly conservative in practice, as we show in numerical experiments. We propose a modified version of the algorithm with tighter upper-confidence bounds, KLOLOP, that leads to better practical performances while retaining the sample complexity bound. Finally, we propose an efficient implementation that significantly improves the time complexity of both algorithms.