# Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interaction

arXiv: https://arxiv.org/abs/2209.13308 · HTML: https://mlsanity.com/paper/2209.13308/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 60 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 30 of 39)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Reward Constrained Policy Optimization](https://mlsanity.com/paper/1805.11074/) (2018)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Responsive Safety in Reinforcement Learning by PID Lagrangian Methods](https://mlsanity.com/paper/2007.03964/) (2020)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems](https://mlsanity.com/paper/1705.01292/) (2017)
- [Lyapunov-based Safe Policy Optimization for Continuous Control](https://mlsanity.com/paper/1901.10031/) (2019)
- [Provably Efficient Safe Exploration via Primal-Dual Policy Optimization](https://mlsanity.com/paper/2003.00534/) (2020)
- [IPO: Interior-point Policy Optimization under Constraints](https://mlsanity.com/paper/1910.09615/) (2019)
- [Learning-based Model Predictive Control for Safe Exploration](https://mlsanity.com/paper/1803.08287/) (2018)
- [Uncertainty-Aware Reinforcement Learning for Collision Avoidance](https://mlsanity.com/paper/1702.01182/) (2017)
- [Safe Reinforcement Learning with Model Uncertainty Estimates](https://mlsanity.com/paper/1810.08700/) (2018)
- [OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World](https://mlsanity.com/paper/1709.07643/) (2017)
- [Learning for Safety-Critical Control with Control Barrier Functions](https://mlsanity.com/paper/1912.10099/) (2019)
- [Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentation](https://mlsanity.com/paper/2202.06558/) (2022)
- [MushroomRL: Simplifying Reinforcement Learning Research](https://mlsanity.com/paper/2001.01102/) (2020)
- [Safe Exploration of State and Action Spaces in Reinforcement Learning](https://mlsanity.com/paper/1402.0560/) (2014)
- [Safe Reinforcement Learning for Legged Locomotion](https://mlsanity.com/paper/2203.02638/) (2022)
- [SAMBA: Safe Model-Based & Active Reinforcement Learning](https://mlsanity.com/paper/2006.09436/) (2020)
- [Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Control](https://mlsanity.com/paper/2011.08421/) (2020)
- [TERP: Reliable Planning in Uneven Outdoor Environments using Deep Reinforcement Learning](https://mlsanity.com/paper/2109.05120/) (2021)

## Built on by (most cited first, 17 of 17)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Safe Reinforcement Learning on the Constraint Manifold: Theory and Applications](https://mlsanity.com/paper/2404.09080/) (2024)
- [Distributionally Robust Constrained Reinforcement Learning under Strong Duality](https://mlsanity.com/paper/2406.15788/) (2024)
- [Efficient Motion Planning for Manipulators with Control Barrier Function-Induced Neural Controller](https://mlsanity.com/paper/2404.01184/) (2024)
- [Hierarchical Policy Blending As Optimal Transport](https://mlsanity.com/paper/2212.01938/) (2022)
- [Bridging the gap between Learning-to-plan, Motion Primitives and Safe Reinforcement Learning](https://mlsanity.com/paper/2408.14063/) (2024)
- [ARMOR: Egocentric Perception for Humanoid Robot Collision Avoidance and Motion Planning](https://mlsanity.com/paper/2412.00396/) (2024)
- [Safety-Driven Deep Reinforcement Learning Framework for Cobots: A Sim2Real Approach](https://mlsanity.com/paper/2407.02231/) (2024)
- [Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning](https://mlsanity.com/paper/2409.12045/) (2024)
- [Bresa: Bio-inspired Reflexive Safe Reinforcement Learning for Contact-Rich Robotic Tasks](https://mlsanity.com/paper/2503.21989/) (2025)
- [Deep reinforcement learning for the dynamic vehicle dispatching problem: An event-based approach](https://mlsanity.com/paper/2307.07508/) (2023)
- [A Safe Exploration Strategy for Model-free Task Adaptation in Safety-constrained Grid Environments](https://mlsanity.com/paper/2408.00997/) (2024)
- [Learning From Failures: Efficient Reinforcement Learning Control with Episodic Memory](https://mlsanity.com/paper/2603.07110/) (2026)
- [V2P-Manip: Learning Dexterous Manipulation from Monocular Human Videos](https://mlsanity.com/paper/2606.16436/) (2026)
- [Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Control](https://mlsanity.com/paper/2606.24010/) (2026)
- [Directional Constraints for Efficient Exploration in Safe Reinforcement Learning](https://mlsanity.com/paper/2607.12784/) (2026)
- [Safety-aware Skill Adaptation for Reinforcement Learning in Dynamic Environments](https://mlsanity.com/paper/2609.11433/) (2026)

## Abstract (arXiv)

Safety is a crucial property of every robotic platform: any control policy should always comply with actuator limits and avoid collisions with the environment and humans. In reinforcement learning, safety is even more fundamental for exploring an environment without causing any damage. While there are many proposed solutions to the safe exploration problem, only a few of them can deal with the complexity of the real world. This paper introduces a new formulation of safe exploration for reinforcement learning of various robotic tasks. Our approach applies to a wide class of robotic platforms and enforces safety even under complex collision constraints learned from data by exploring the tangent space of the constraint manifold. Our proposed approach achieves state-of-the-art performance in simulated high-dimensional and dynamic tasks while avoiding collisions with the environment. We show safe real-world deployment of our learned controller on a TIAGo++ robot, achieving remarkable performance in manipulation and human-robot interaction tasks.