mlsanityData to 7 Oct 2026

SMARLA: A Safety Monitoring Approach for Deep Reinforcement Learning Agents

arXiv 2308.02594 ↗First posted 3 Aug 2023cs.LG cs.AI cs.SEcited by 9

Not on the map. Why?

Abstract (arXiv)

Deep Reinforcement Learning (DRL) has made significant advancements in various fields, such as autonomous driving, healthcare, and robotics, by enabling agents to learn optimal policies through interactions with their environments. However, the application of DRL in safety-critical domains presents challenges, particularly concerning the safety of the learned policies. DRL agents, which are focused on maximizing rewards, may select unsafe actions, leading to safety violations. Runtime safety monitoring is thus essential to ensure the safe operation of these agents, especially in unpredictable and dynamic environments. This paper introduces SMARLA, a black-box safety monitoring approach specifically designed for DRL agents. SMARLA utilizes machine learning to predict safety violations by observing the agent's behavior during execution. The approach is based on Q-values, which reflect the expected reward for taking actions in specific states. SMARLA employs state abstraction to reduce the complexity of the state space, enhancing the predictive capabilities of the monitoring model. Such abstraction enables the early detection of unsafe states, allowing for the implementation of corrective and preventive measures before incidents occur. We quantitatively and qualitatively validated SMARLA on three well-known case studies widely used in DRL research. Empirical results reveal that SMARLA is accurate at predicting safety violations, with a low false positive rate, and can predict violations at an early stage, approximately halfway through the execution of the agent, before violations occur. We also discuss different decision criteria, based on confidence intervals of the predicted violation probabilities, to trigger safety mechanisms aiming at a trade-off between early detection and low false positive rates.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 47 of its 102 references that are in the map. First posted on arXiv 3 Aug 2023.

Builds on

47 of 102 references.

References · 47

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  3. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  4. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  5. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  6. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  8. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  9. 2017Safe Reinforcement Learning via Shieldingcited by 427
  10. 2019Challenges of Real-World Reinforcement Learningcited by 282
  11. 2017Noisy Networks for Explorationcited by 262
  12. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
35 more
  1. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  2. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  3. 2017Action Branching Architectures for Deep Reinforcement Learningcited by 90
  4. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 73
  5. 2019Discretizing Continuous Action Space for On-Policy Optimizationcited by 65
  6. 2019Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Controlcited by 60
  7. 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 60
  8. 2019Misbehaviour Prediction for Autonomous Driving Systemscited by 55
  9. 2018Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Controlcited by 54
  10. 2021Memory-based Deep Reinforcement Learning for POMDPscited by 53
  11. 2019Outside the Box: Abstraction-Based Monitoring of Neural Networkscited by 36
  12. 2018Runtime Monitoring Neuron Activation Patternscited by 34
  13. 2020Stealing Deep Reinforcement Learning Models for Fun and Profitcited by 24
  14. 2021Human-robot collaboration and machine learning: a systematic review of recent researchcited by 20
  15. 2022Online Shielding for Reinforcement Learningcited by 19
  16. 2021Black-Box Testing of Deep Neural Networks Through Test Case Diversitycited by 18
  17. 2021DeepGuard: A Framework for Safeguarding Autonomous Driving Systems from Inconsistent Behaviorcited by 17
  18. 2020SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measurecited by 16
  19. 2017Deep Predictive Models for Collision Risk Assessment in Autonomous Drivingcited by 15
  20. 2021Benchmarking Safety Monitors for Image Classifiers with Machine Learningcited by 14
  21. 2019Adversarial Exploitation of Policy Imitationcited by 13
  22. 2022Search-Based Testing of Reinforcement Learningcited by 12
  23. 2022A Search-Based Testing Approach for Deep Reinforcement Learning Agentscited by 12
  24. 2021Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoringcited by 11
  25. 2023DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networkscited by 11
  26. 2022Comparing Model-free and Model-based Algorithms for Offline Reinforcement Learningcited by 9
  27. 2023Safe Deep Reinforcement Learning by Verifying Task-Level Propertiescited by 9
  28. 2019Sequential Triggers for Watermarking of Deep Reinforcement Learning Policiescited by 7
  29. 2021Fault-Tolerant Perception for Automated Driving A Lightweight Monitoring Approachcited by 5
  30. 2023Mosaic: Model-based Safety Analysis Framework for AI-enabled Cyber-Physical Systemscited by 5
  31. 2019Sample-Efficient Model-Free Reinforcement Learning with Off-Policy Criticscited by 3
  32. 2022Safe Exploration Method for Reinforcement Learning under Existence of Disturbancecited by 3
  33. 2023Comparing NARS and Reinforcement Learning: An Analysis of ONA and Q-Learning Algorithmscited by 3
  34. 2021On Assessing The Safety of Reinforcement Learning algorithms Using Formal Methodscited by 2
  35. 2022Towards Runtime Monitoring of Complex System Requirements for Autonomous Driving Functionscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 5.

9 papers.

  1. 2024Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forwardcited by 5
  2. 2024System Safety Monitoring of Learned Components Using Temporal Metric Forecastingcited by 4
  3. 2025Q-SafeML: Safety Assessment of Quantum Machine Learning via Quantum Distance Metricscited by 1
  4. 2026Learning Contextual Runtime Monitors for Safe AI-Based Autonomycited by 1
  5. 2025Learning-Based Testing for Deep Learning: Enhancing Model Robustness with Adversarial Input Prioritizationcited by 0
  6. 2026COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Managementcited by 0
  7. 2026COOL-MC: Verifying and Explaining RL Policies for Multi-bridge Network Maintenancecited by 0
  8. 2026Evaluating Fuzz Testing for Reinforcement Learning Agentscited by 0
  9. 2026A MARL Centered Reference Architecture for Large Language Model Augmentation in Smart Manufacturingcited by 0