# Improving the Exploration of Deep Reinforcement Learning in Continuous Domains using Planning for Policy Search

arXiv: https://arxiv.org/abs/2010.12974 · HTML: https://mlsanity.com/paper/2010.12974/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2020.

## Built on by (most cited first, 1 of 1)

- [Search Inspired Exploration in Reinforcement Learning](https://mlsanity.com/paper/2602.00460/) (2026)

## Abstract (arXiv)

Local policy search is performed by most Deep Reinforcement Learning (D-RL) methods, which increases the risk of getting trapped in a local minimum. Furthermore, the availability of a simulation model is not fully exploited in D-RL even in simulation-based training, which potentially decreases efficiency. To better exploit simulation models in policy search, we propose to integrate a kinodynamic planner in the exploration strategy and to learn a control policy in an offline fashion from the generated environment interactions. We call the resulting model-based reinforcement learning method PPS (Planning for Policy Search). We compare PPS with state-of-the-art D-RL methods in typical RL settings including underactuated systems. The comparison shows that PPS, guided by the kinodynamic planner, collects data from a wider region of the state space. This generates training data that helps PPS discover better policies.