# PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning

arXiv: https://arxiv.org/abs/1710.03937 · HTML: https://mlsanity.com/paper/1710.03937/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2017.

## Built on by (most cited first, 30 of 69)

- [Habitat 2.0: Training Home Assistants to Rearrange their Habitat](https://mlsanity.com/paper/2106.14405/) (2021)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023)
- [Rapid Exploration for Open-World Navigation with Latent Goal Models](https://mlsanity.com/paper/2104.05859/) (2021)
- [Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexity](https://mlsanity.com/paper/2210.09579/) (2022)
- [ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints](https://mlsanity.com/paper/2202.11271/) (2022)
- [Retrospectives on the Embodied AI Workshop](https://mlsanity.com/paper/2210.06849/) (2022)
- [Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation](https://mlsanity.com/paper/2209.10780/) (2022)
- [Offline Reinforcement Learning for Visual Navigation](https://mlsanity.com/paper/2212.08244/) (2022)
- [The Logical Options Framework](https://mlsanity.com/paper/2102.12571/) (2021)
- [Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization](https://mlsanity.com/paper/2109.14549/) (2021)
- [Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-End](https://mlsanity.com/paper/2310.03581/) (2023)
- [Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigation](https://mlsanity.com/paper/2104.03111/) (2021)
- [Arena-Rosnav: Towards Deployment of Deep-Reinforcement-Learning-Based Obstacle Avoidance into Conventional Autonomous Navigation Systems](https://mlsanity.com/paper/2104.03616/) (2021)
- [Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation](https://mlsanity.com/paper/2109.00525/) (2021)
- [Data Representativity for Machine Learning and AI Systems](https://mlsanity.com/paper/2203.04706/) (2022)
- [Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization](https://mlsanity.com/paper/2209.00347/) (2022)
- [Learning Speed Adaptation for Flight in Clutter](https://mlsanity.com/paper/2403.04586/) (2024)
- [Coarse-to-Fine Q-attention with Learned Path Ranking](https://mlsanity.com/paper/2204.01571/) (2022)
- [Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2506.07744/) (2025)
- [Hierarchies of Planning and Reinforcement Learning for Robot Navigation](https://mlsanity.com/paper/2109.11178/) (2021)
- [Combining Planning and Learning of Behavior Trees for Robotic Assembly](https://mlsanity.com/paper/2103.09036/) (2021)
- [Rule-Based Reinforcement Learning for Efficient Robot Navigation with Space Reduction](https://mlsanity.com/paper/2104.07282/) (2021)
- [SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2105.06228/) (2021)
- [PathRL: An End-to-End Path Generation Method for Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/2310.13295/) (2023)
- [Connecting Deep-Reinforcement-Learning-based Obstacle Avoidance with Conventional Global Planners using Waypoint Generators](https://mlsanity.com/paper/2104.03663/) (2021)
- [MMD-MIX: Value Function Factorisation with Maximum Mean Discrepancy for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.11652/) (2021)
- [Model-free Neural Lyapunov Control for Safe Robot Navigation](https://mlsanity.com/paper/2203.01190/) (2022)
- [Control Transformer: Robot Navigation in Unknown Environments through PRM-Guided Return-Conditioned Sequence Modeling](https://mlsanity.com/paper/2211.06407/) (2022)

## Abstract (arXiv)

We present PRM-RL, a hierarchical method for long-range navigation task completion that combines sampling based path planning with reinforcement learning (RL). The RL agents learn short-range, point-to-point navigation policies that capture robot dynamics and task constraints without knowledge of the large-scale topology. Next, the sampling-based planners provide roadmaps which connect robot configurations that can be successfully navigated by the RL agent. The same RL agents are used to control the robot under the direction of the planning, enabling long-range navigation. We use the Probabilistic Roadmaps (PRMs) for the sampling-based planner. The RL agents are constructed using feature-based and deep neural net policies in continuous state and action spaces. We evaluate PRM-RL, both in simulation and on-robot, on two navigation tasks with non-trivial robot dynamics: end-to-end differential drive indoor navigation in office environments, and aerial cargo delivery in urban environments with load displacement constraints. Our results show improvement in task completion over both RL agents on their own and traditional sampling-based planners. In the indoor navigation task, PRM-RL successfully completes up to 215 m long trajectories under noisy sensor conditions, and the aerial cargo delivery completes flights over 1000 m without violating the task constraints in an environment 63 million times larger than used in training.