# RL-RRT: Kinodynamic Motion Planning via Learning Reachability Estimators from RL Policies

arXiv: https://arxiv.org/abs/1907.04799 · HTML: https://mlsanity.com/paper/1907.04799/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2019.

## Built on by (most cited first, 20 of 20)

- [SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policies](https://mlsanity.com/paper/2109.13602/) (2021)
- [Hierarchies of Planning and Reinforcement Learning for Robot Navigation](https://mlsanity.com/paper/2109.11178/) (2021)
- [MPC-MPNet: Model-Predictive Motion Planning Networks for Fast, Near-Optimal Planning under Kinodynamic Constraints](https://mlsanity.com/paper/2101.06798/) (2021)
- [Model-free Neural Lyapunov Control for Safe Robot Navigation](https://mlsanity.com/paper/2203.01190/) (2022)
- [A Survey on the Integration of Machine Learning with Sampling-based Motion Planning](https://mlsanity.com/paper/2211.08368/) (2022)
- [Intention-Aware Navigation in Crowds with Extended-Space POMDP Planning](https://mlsanity.com/paper/2206.10028/) (2022)
- [R×R: Rapid eXploration for Reinforcement Learning via Sampling-based Reset Distributions and Imitation Pre-training](https://mlsanity.com/paper/2401.15484/) (2024)
- [Train-Once Plan-Anywhere Kinodynamic Motion Planning via Diffusion Trees](https://mlsanity.com/paper/2508.21001/) (2025)
- [Estimation Error Correction in Deep Reinforcement Learning for Deterministic Actor-Critic Methods](https://mlsanity.com/paper/2109.10736/) (2021)
- [Robot Navigation with Reinforcement Learned Path Generation and Fine-Tuned Motion Control](https://mlsanity.com/paper/2210.10639/) (2022)
- [Learning from Demonstration without Demonstrations](https://mlsanity.com/paper/2106.09203/) (2021)
- [RoboArm-NMP: a Learning Environment for Neural Motion Planning](https://mlsanity.com/paper/2405.16335/) (2024)
- [Fast End-to-End Generation of Belief Space Paths for Minimum Sensing Navigation](https://mlsanity.com/paper/2409.12902/) (2024)
- [Zonal RL-RRT: Integrated RL-RRT Path Planning with Collision Probability and Zone Connectivity](https://mlsanity.com/paper/2410.24205/) (2024)
- [SIL-RRT*: Learning Sampling Distribution through Self Imitation Learning](https://mlsanity.com/paper/2411.17293/) (2024)
- [Self-Supervised Learning-Based Path Planning and Obstacle Avoidance Using PPO and B-Splines in Unknown Environments](https://mlsanity.com/paper/2412.02176/) (2024)
- [SafeMove-RL: A Certifiable Reinforcement Learning Framework for Dynamic Motion Constraints in Trajectory Planning](https://mlsanity.com/paper/2505.12648/) (2025)
- [Learning Reach-Avoid Task with Reinforcement Learning: Vectorized Simulation and Benchmark](https://mlsanity.com/paper/2607.15935/) (2026)
- [Neurosymbolic Reasoning with Incremental Knowledge for Sample Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2608.02993/) (2026)
- [Learning-Based Dynamic Obstacle Avoidance for a UAV Using Only Three Range Sensors](https://mlsanity.com/paper/2609.14426/) (2026)

## Abstract (arXiv)

This paper addresses two challenges facing sampling-based kinodynamic motion planning: a way to identify good candidate states for local transitions and the subsequent computationally intractable steering between these candidate states. Through the combination of sampling-based planning, a Rapidly Exploring Randomized Tree (RRT) and an efficient kinodynamic motion planner through machine learning, we propose an efficient solution to long-range planning for kinodynamic motion planning. First, we use deep reinforcement learning to learn an obstacle-avoiding policy that maps a robot's sensor observations to actions, which is used as a local planner during planning and as a controller during execution. Second, we train a reachability estimator in a supervised manner, which predicts the RL policy's time to reach a state in the presence of obstacles. Lastly, we introduce RL-RRT that uses the RL policy as a local planner, and the reachability estimator as the distance function to bias tree-growth towards promising regions. We evaluate our method on three kinodynamic systems, including physical robot experiments. Results across all three robots tested indicate that RL-RRT outperforms state of the art kinodynamic planners in efficiency, and also provides a shorter path finish time than a steering function free method. The learned local planner policy and accompanying reachability estimator demonstrate transferability to the previously unseen experimental environments, making RL-RRT fast because the expensive computations are replaced with simple neural network inference. Video: https://youtu.be/dDMVMTOI8KY