# Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments

arXiv: https://arxiv.org/abs/2010.11940 · HTML: https://mlsanity.com/paper/2010.11940/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2020.

## Built on by (most cited first, 23 of 23)

- [Learning to Synthesize Programs as Interpretable and Generalizable Policies](https://mlsanity.com/paper/2108.13643/) (2021)
- [Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks](https://mlsanity.com/paper/2405.01534/) (2024)
- [Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulation](https://mlsanity.com/paper/2203.09749/) (2022)
- [Learning Deformable Object Manipulation from Expert Demonstrations](https://mlsanity.com/paper/2207.10148/) (2022)
- [SRL-VIC: A Variable Stiffness-Based Safe Reinforcement Learning for Contact-Rich Robotic Tasks](https://mlsanity.com/paper/2406.13744/) (2024)
- [P-RAG: Progressive Retrieval Augmented Generation For Planning on Embodied Everyday Task](https://mlsanity.com/paper/2409.11279/) (2024)
- [Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation](https://mlsanity.com/paper/2111.06383/) (2021)
- [Waypoint-Based Reinforcement Learning for Robot Manipulation Tasks](https://mlsanity.com/paper/2403.13281/) (2024)
- [Jacta: A Versatile Planner for Learning Dexterous and Whole-body Manipulation](https://mlsanity.com/paper/2408.01258/) (2024)
- [Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation](https://mlsanity.com/paper/2412.02676/) (2024)
- [Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy](https://mlsanity.com/paper/2508.13103/) (2025)
- [PLANRL: A Motion Planning and Imitation Learning Framework to Bootstrap Reinforcement Learning](https://mlsanity.com/paper/2408.04054/) (2024)
- [Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery](https://mlsanity.com/paper/2204.13906/) (2022)
- [Reinforcement Learning with Prior Policy Guidance for Motion Planning of Dual-Arm Free-Floating Space Robot](https://mlsanity.com/paper/2209.01434/) (2022)
- [A Learning System for Motion Planning of Free-Float Dual-Arm Space Manipulator towards Non-Cooperative Object](https://mlsanity.com/paper/2207.02464/) (2022)
- [Decoupling Skill Learning from Robotic Control for Generalizable Object Manipulation](https://mlsanity.com/paper/2303.04016/) (2023)
- [Efficient Learning of High Level Plans from Play](https://mlsanity.com/paper/2303.09628/) (2023)
- [Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control](https://mlsanity.com/paper/2509.06201/) (2025)
- [GenDexHand: Generative Simulation for Dexterous Hands](https://mlsanity.com/paper/2511.01791/) (2025)
- [Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport](https://mlsanity.com/paper/2210.15908/) (2022)
- [RoboArm-NMP: a Learning Environment for Neural Motion Planning](https://mlsanity.com/paper/2405.16335/) (2024)
- [Hybrid Robot Learning for Automatic Robot Motion Planning in Manufacturing](https://mlsanity.com/paper/2502.19340/) (2025)
- [Re4MPC: Reactive Nonlinear MPC for Multi-model Motion Planning via Deep Reinforcement Learning](https://mlsanity.com/paper/2506.08344/) (2025)

## Abstract (arXiv)

Deep reinforcement learning (RL) agents are able to learn contact-rich manipulation tasks by maximizing a reward signal, but require large amounts of experience, especially in environments with many obstacles that complicate exploration. In contrast, motion planners use explicit models of the agent and environment to plan collision-free paths to faraway goals, but suffer from inaccurate models in tasks that require contacts with the environment. To combine the benefits of both approaches, we propose motion planner augmented RL (MoPA-RL) which augments the action space of an RL agent with the long-horizon planning capabilities of motion planners. Based on the magnitude of the action, our approach smoothly transitions between directly executing the action and invoking a motion planner. We evaluate our approach on various simulated manipulation tasks and compare it to alternative action spaces in terms of learning efficiency and safety. The experiments demonstrate that MoPA-RL increases learning efficiency, leads to a faster exploration, and results in safer policies that avoid collisions with the environment. Videos and code are available at https://clvrai.com/mopa-rl .