# Provably Safe Deep Reinforcement Learning for Robotic Manipulation in Human Environments

arXiv: https://arxiv.org/abs/2205.06311 · HTML: https://mlsanity.com/paper/2205.06311/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 12 May 2022.

## Builds on (most cited first, 15 of 15)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specifications](https://mlsanity.com/paper/1807.06333/) (2018)
- [OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World](https://mlsanity.com/paper/1709.07643/) (2017)
- [Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning](https://mlsanity.com/paper/1912.11032/) (2019)
- [Exploration via Hindsight Goal Generation](https://mlsanity.com/paper/1906.04279/) (2019)
- [Verifiably Safe Exploration for End-to-End Reinforcement Learning](https://mlsanity.com/paper/2007.01223/) (2020)
- [Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Control](https://mlsanity.com/paper/2011.08421/) (2020)
- [Robotic Table Tennis with Model-Free Reinforcement Learning](https://mlsanity.com/paper/2003.14398/) (2020)
- [Online Shielding for Stochastic Systems](https://mlsanity.com/paper/2012.09539/) (2020)
- [Complex Robotic Manipulation via Graph-Based Hindsight Goal Generation](https://mlsanity.com/paper/2007.13486/) (2020)

## Built on by (most cited first, 18 of 18)

- [Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking](https://mlsanity.com/paper/2205.06750/) (2022)
- [Provably Safe Reinforcement Learning via Action Projection using Reachability Analysis and Polynomial Zonotopes](https://mlsanity.com/paper/2210.10691/) (2022)
- [Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interaction](https://mlsanity.com/paper/2209.13308/) (2022)
- [Safe Reinforcement Learning on the Constraint Manifold: Theory and Applications](https://mlsanity.com/paper/2404.09080/) (2024)
- [Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulation](https://mlsanity.com/paper/2401.08957/) (2024)
- [Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2404.16879/) (2024)
- [RAIL: Reachability-Aided Imitation Learning for Safe Policy Execution](https://mlsanity.com/paper/2409.19190/) (2024)
- [From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies](https://mlsanity.com/paper/2511.06385/) (2025)
- [Safe Reinforcement Learning for Power System Control: A Review](https://mlsanity.com/paper/2407.00681/) (2024)
- [Safety-Driven Deep Reinforcement Learning Framework for Cobots: A Sim2Real Approach](https://mlsanity.com/paper/2407.02231/) (2024)
- [A Design Co-Pilot for Task-Tailored Manipulators](https://mlsanity.com/paper/2509.13077/) (2025)
- [X-Safe: Embodiment-Agnostic Safeguarding for Robot Arm Manipulation Policies via Action Masking](https://mlsanity.com/paper/2606.22278/) (2026)
- [Enhancing Offline Model-Based RL via Active Model Selection: A Bayesian Optimization Perspective](https://mlsanity.com/paper/2502.11480/) (2025)
- [Leveraging Analytic Gradients in Provably Safe Reinforcement Learning](https://mlsanity.com/paper/2506.01665/) (2025)
- [A Parametric Model for Near-Optimal Online Synthesis with Robust Reach-Avoid Guarantees](https://mlsanity.com/paper/2504.01006/) (2025)
- [Vision-Based Safe Human-Robot Collaboration with Uncertainty Guarantees](https://mlsanity.com/paper/2604.15221/) (2026)
- [No Free Checker: A Survey of Verifiers for Robot Policies](https://mlsanity.com/paper/2609.09250/) (2026)
- [Learning from Runtime Feedback through Failure-Bank Self-Evolution for Vision-Language-Action Models](https://mlsanity.com/paper/2609.39820/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (RL) has shown promising results in the motion planning of manipulators. However, no method guarantees the safety of highly dynamic obstacles, such as humans, in RL-based manipulator control. This lack of formal safety assurances prevents the application of RL for manipulators in real-world human environments. Therefore, we propose a shielding mechanism that ensures ISO-verified human safety while training and deploying RL algorithms on manipulators. We utilize a fast reachability analysis of humans and manipulators to guarantee that the manipulator comes to a complete stop before a human is within its range. Our proposed method guarantees safety and significantly improves the RL performance by preventing episode-ending collisions. We demonstrate the performance of our proposed method in simulation using human motion capture data.