# POLICEd RL: Learning Closed-Loop Robot Control Policies with Provable Satisfaction of Hard Constraints

arXiv: https://arxiv.org/abs/2403.13297 · HTML: https://mlsanity.com/paper/2403.13297/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2024.

## Built on by (most cited first, 5 of 5)

- [RAIL: Reachability-Aided Imitation Learning for Safe Policy Execution](https://mlsanity.com/paper/2409.19190/) (2024)
- [Provably-Safe Neural Network Training Using Hybrid Zonotope Reachability Analysis](https://mlsanity.com/paper/2501.13023/) (2025)
- [X-Safe: Embodiment-Agnostic Safeguarding for Robot Arm Manipulation Policies via Action Masking](https://mlsanity.com/paper/2606.22278/) (2026)
- [mPOLICE: Provable Enforcement of Multi-Region Affine Constraints in Deep Neural Networks](https://mlsanity.com/paper/2502.02434/) (2025)
- [ShardNet: Training Neural Controllers with Hard, Non-Convex Constraints](https://mlsanity.com/paper/2606.30935/) (2026)

## Abstract (arXiv)

In this paper, we seek to learn a robot policy guaranteed to satisfy state constraints. To encourage constraint satisfaction, existing RL algorithms typically rely on Constrained Markov Decision Processes and discourage constraint violations through reward shaping. However, such soft constraints cannot offer verifiable safety guarantees. To address this gap, we propose POLICEd RL, a novel RL algorithm explicitly designed to enforce affine hard constraints in closed-loop with a black-box environment. Our key insight is to force the learned policy to be affine around the unsafe set and use this affine region as a repulsive buffer to prevent trajectories from violating the constraint. We prove that such policies exist and guarantee constraint satisfaction. Our proposed framework is applicable to both systems with continuous and discrete state and action spaces and is agnostic to the choice of the RL training algorithm. Our results demonstrate the capacity of POLICEd RL to enforce hard constraints in robotic tasks while significantly outperforming existing methods.