# A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks

arXiv: https://arxiv.org/abs/2412.04153 · HTML: https://mlsanity.com/paper/2412.04153/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 23 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 19 of 19)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Reward Constrained Policy Optimization](https://mlsanity.com/paper/1805.11074/) (2018)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Responsive Safety in Reinforcement Learning by PID Lagrangian Methods](https://mlsanity.com/paper/2007.03964/) (2020)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018)
- [OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Research](https://mlsanity.com/paper/2305.09304/) (2023)
- [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024)
- [Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentation](https://mlsanity.com/paper/2202.06558/) (2022)
- [Safe Reinforcement Learning via Shielding under Partial Observability](https://mlsanity.com/paper/2204.00755/) (2022)
- [SafeDreamer: Safe Reinforcement Learning with World Models](https://mlsanity.com/paper/2307.07176/) (2023)
- [Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians](https://mlsanity.com/paper/2102.13073/) (2021)
- [Probabilistic model predictive safety certification for learning-based control](https://mlsanity.com/paper/1906.10417/) (2019)
- [MAMPS: Safe Multi-Agent Reinforcement Learning via Model Predictive Shielding](https://mlsanity.com/paper/1910.12639/) (2019)
- [Benchmarking Reinforcement Learning Techniques for Autonomous Navigation](https://mlsanity.com/paper/2210.04839/) (2022)
- [Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios](https://mlsanity.com/paper/2210.02300/) (2022)
- [Safe Multi-Agent Reinforcement Learning for Behavior-Based Cooperative Navigation](https://mlsanity.com/paper/2312.12861/) (2023)

## Built on by (most cited first, 5 of 5)

- [Safety Must Precede the Deployment of Open-Ended AI](https://mlsanity.com/paper/2502.04512/) (2025)
- [Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigation](https://mlsanity.com/paper/2504.21643/) (2025)
- [Perturbation-mitigated USV Navigation with Distributionally Robust Reinforcement Learning](https://mlsanity.com/paper/2512.00030/) (2025)
- [Constraint-Aware Reinforcement Learning via Adaptive Action Scaling](https://mlsanity.com/paper/2510.11491/) (2025)
- [Probabilistic Safety Guarantee for Stochastic Control Systems Using Average Reward MDPs](https://mlsanity.com/paper/2511.08419/) (2025)

## Abstract (arXiv)

Reinforcement learning (RL) has been successfully applied to a variety of robotics applications, where it outperforms classical methods. However, the safety aspect of RL and the transfer to the real world remain an open challenge. A prominent field for tackling this challenge and ensuring the safety of the agents during training and execution is safe reinforcement learning. Safe RL can be achieved through constrained RL and safe exploration approaches. The former learns the safety constraints over the course of training to achieve a safe behavior by the end of training, at the cost of high number of collisions at earlier stages of the training. The latter offers robust safety by enforcing the safety constraints as hard constraints, which prevents collisions but hinders the exploration of the RL agent, resulting in lower rewards and poor performance. To overcome those drawbacks, we propose a novel safety shield, that combines the robustness of the optimization-based controllers with the long prediction capabilities of the RL agents, allowing the RL agent to adaptively tune the parameters of the controller. Our approach is able to improve the exploration of the RL agents for navigation tasks, while minimizing the number of collisions. Experiments in simulation show that our approach outperforms state-of-the-art baselines in the reached goals-to-collisions ratio in different challenging environments. The goals-to-collisions ratio metrics emphasizes the importance of minimizing the number of collisions, while learning to accomplish the task. Our approach achieves a higher number of reached goals compared to the classic safety shields and fewer collisions compared to constrained RL approaches. Finally, we demonstrate the performance of the proposed method in a real-world experiment.