Reinfier and Reintrainer: Verification and Interpretation-Driven Safe Deep Reinforcement Learning Frameworks

arXiv 2410.15127 ↗First posted 19 Oct 2024cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Ensuring verifiable and interpretable safety of deep reinforcement learning (DRL) is crucial for its deployment in real-world applications. Existing approaches like verification-in-the-loop training, however, face challenges such as difficulty in deployment, inefficient training, lack of interpretability, and suboptimal performance in property satisfaction and reward performance. In this work, we propose a novel verification-driven interpretation-in-the-loop framework Reintrainer to develop trustworthy DRL models, which are guaranteed to meet the expected constraint properties. Specifically, in each iteration, this framework measures the gap between the on-training model and predefined properties using formal verification, interprets the contribution of each input feature to the model's output, and then generates the training strategy derived from the on-the-fly measure results, until all predefined properties are proven. Additionally, the low reusability of existing verifiers and interpreters motivates us to develop Reinfier, a general and fundamental tool within Reintrainer for DRL verification and interpretation. Reinfier features breakpoints searching and verification-driven interpretation, associated with a concise constraint-encoding language DRLP. Evaluations demonstrate that Reintrainer outperforms the state-of-the-art on six public benchmarks in both performance and property guarantees. Our framework can be accessed at https://github.com/Kurayuri/Reinfier.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 47 references that are in the map. First posted on arXiv 19 Oct 2024.

Builds on

18 of 47 references.

References · 18

2017Proximal Policy Optimization Algorithmscited by 15,215
2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
2015Continuous control with deep reinforcement learningcited by 3,407
2013Playing Atari with Deep Reinforcement Learningcited by 3,274
2016OpenAI Gymcited by 1,736
2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
2018A Survey Of Methods For Explaining Black Box Modelscited by 973
2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
2018Feature Denoising for Improving Adversarial Robustnesscited by 314
2020A Survey on Neural Network Interpretabilitycited by 291
2020Projection-Based Constrained Policy Optimizationcited by 203
2018Applications of Deep Reinforcement Learning in Communications and Networking: A Surveycited by 167

6 more

2020Learning to be Safe: Deep RL with a Safety Criticcited by 116
2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 58
2021Towards Scalable Verification of Deep Reinforcement Learningcited by 33
2022Verifying Learning-Based Robotic Navigation Systemscited by 33
2021DNNV: A Framework for Deep Neural Network Verificationcited by 23
2021Robust Models Are More Interpretable Because Attributions Look Normalcited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way