# Bridging the gap between Learning-to-plan, Motion Primitives and Safe Reinforcement Learning

arXiv: https://arxiv.org/abs/2408.14063 · HTML: https://mlsanity.com/paper/2408.14063/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 45 of its 63 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 30 of 45)

- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Reward Constrained Policy Optimization](https://mlsanity.com/paper/1805.11074/) (2018)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Responsive Safety in Reinforcement Learning by PID Lagrangian Methods](https://mlsanity.com/paper/2007.03964/) (2020)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [Projection-Based Constrained Policy Optimization](https://mlsanity.com/paper/2010.03152/) (2020)
- [A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems](https://mlsanity.com/paper/1705.01292/) (2017)
- [Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models](https://mlsanity.com/paper/2308.01557/) (2023)
- [Lyapunov-based Safe Policy Optimization for Continuous Control](https://mlsanity.com/paper/1901.10031/) (2019)
- [Provably Efficient Safe Exploration via Primal-Dual Policy Optimization](https://mlsanity.com/paper/2003.00534/) (2020)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [IPO: Interior-point Policy Optimization under Constraints](https://mlsanity.com/paper/1910.09615/) (2019)
- [OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World](https://mlsanity.com/paper/1709.07643/) (2017)
- [Learning for Safety-Critical Control with Control Barrier Functions](https://mlsanity.com/paper/1912.10099/) (2019)
- [Motion Planning Networks: Bridging the Gap Between Learning-based and Classical Motion Planners](https://mlsanity.com/paper/1907.06013/) (2019)
- [Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model](https://mlsanity.com/paper/2401.10700/) (2024)
- [Safe Reinforcement Learning Using Robust Control Barrier Functions](https://mlsanity.com/paper/2110.05415/) (2021)
- [Dynamic Movement Primitives in Robotics: A Tutorial Survey](https://mlsanity.com/paper/2102.03861/) (2021)
- [Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Risk](https://mlsanity.com/paper/2206.04436/) (2022)
- [Safe Exploration of State and Action Spaces in Reinforcement Learning](https://mlsanity.com/paper/1402.0560/) (2014)
- [Learning to Play Table Tennis From Scratch using Muscular Robots](https://mlsanity.com/paper/2006.05935/) (2020)
- [Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Control](https://mlsanity.com/paper/2011.08421/) (2020)
- [Safe Reinforcement Learning Using Black-Box Reachability Analysis](https://mlsanity.com/paper/2204.07417/) (2022)
- [Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion](https://mlsanity.com/paper/2002.09676/) (2020)
- [Differentiable Trust Region Layers for Deep Reinforcement Learning](https://mlsanity.com/paper/2101.09207/) (2021)
- [ProDMPs: A Unified Perspective on Dynamic and Probabilistic Movement Primitives](https://mlsanity.com/paper/2210.01531/) (2022)
- [Specializing Versatile Skill Libraries using Local Mixture of Experts](https://mlsanity.com/paper/2112.04216/) (2021)

## Built on by (most cited first, 4 of 4)

- [Motion Planning Diffusion: Learning and Adapting Robot Motion Planning with Diffusion Models](https://mlsanity.com/paper/2412.19948/) (2024)
- [BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning](https://mlsanity.com/paper/2506.06072/) (2025)
- [TOP-ERL: Transformer-based Off-Policy Episodic Reinforcement Learning](https://mlsanity.com/paper/2410.09536/) (2024)
- [MoRe-ERL: Learning Motion Residuals using Episodic Reinforcement Learning](https://mlsanity.com/paper/2508.01409/) (2025)

## Abstract (arXiv)

Trajectory planning under kinodynamic constraints is fundamental for advanced robotics applications that require dexterous, reactive, and rapid skills in complex environments. These constraints, which may represent task, safety, or actuator limitations, are essential for ensuring the proper functioning of robotic platforms and preventing unexpected behaviors. Recent advances in kinodynamic planning demonstrate that learning-to-plan techniques can generate complex and reactive motions under intricate constraints. However, these techniques necessitate the analytical modeling of both the robot and the entire task, a limiting assumption when systems are extremely complex or when constructing accurate task models is prohibitive. This paper addresses this limitation by combining learning-to-plan methods with reinforcement learning, resulting in a novel integration of black-box learning of motion primitives and optimization. We evaluate our approach against state-of-the-art safe reinforcement learning methods, showing that our technique, particularly when exploiting task structure, outperforms baseline methods in challenging scenarios such as planning to hit in robot air hockey. This work demonstrates the potential of our integrated approach to enhance the performance and safety of robots operating under complex kinodynamic constraints.