mlsanityData to 7 Oct 2026

COOL-MC: Verifying and Explaining RL Policies for Multi-bridge Network Maintenance

arXiv 2603.07546 ↗First posted 8 Mar 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Aging bridge networks require proactive, verifiable, and interpretable maintenance strategies, yet reinforcement learning (RL) policies trained solely on reward signals provide no formal safety guarantees and remain opaque to infrastructure managers. We demonstrate COOL-MC as a tool for verifying and explaining RL policies for multi-bridge network maintenance, building on a single-bridge Markov decision process (MDP) from the literature and extending it to a parallel network of three heterogeneous bridges with a shared periodic budget constraint, encoded in the PRISM modeling language. We train an RL agent on this MDP and apply probabilistic model checking and explainability methods to the induced discrete-time Markov chain (DTMC) that arises from the interaction between the learned policy and the underlying MDP. Probabilistic model checking reveals that the trained policy has a safety-violation probability of 3.5% over the planning horizon, being slightly above the theoretical minimum of 0% and indicating the suboptimality of the learned policy, noting that these results are based on artificially constructed transition probabilities and deterioration rates rather than real-world data, so absolute performance figures should be interpreted with caution. The explainability analysis further reveals, for instance, a systematic bias in the trained policy toward the state of bridge 1 over the remaining bridges in the network. These results demonstrate COOL-MC's ability to provide formal, interpretable, and practical analysis of RL maintenance policies.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 52 of its 135 references that are in the map. First posted on arXiv 8 Mar 2026.

Builds on

52 of 135 references.

References · 52

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  4. 2016Reinforcement Learning With Temporal Logic Rewardscited by 114
  5. 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
  6. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 74
  7. 2017Environment-Independent Task Specifications via GLTLcited by 67
  8. 2016Q-Learning for Robust Satisfaction of Signal Temporal Logic Specificationscited by 65
  9. 2020The Probabilistic Model Checker Stormcited by 64
  10. 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 53
  11. 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
  12. 2020Neurosymbolic Reinforcement Learning with Formally Verified Explorationcited by 49
38 more
  1. 2019An Inductive Synthesis Framework for Verifiable Reinforcement Learningcited by 39
  2. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  3. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 33
  4. 2018Managing engineering systems with large state and action spaces through deep reinforcement learningcited by 32
  5. 2020Verifiably Safe Exploration for End-to-End Reinforcement Learningcited by 32
  6. 2015Safety-Constrained Reinforcement Learning for MDPscited by 31
  7. 2020Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfactioncited by 22
  8. 2020Tractable Reinforcement Learning of Signal Temporal Logic Objectivescited by 21
  9. 2022COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checkingcited by 19
  10. 2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 17
  11. 2015Permissive Controller Synthesis for Probabilistic Systemscited by 16
  12. 2020Probabilistic Guarantees for Safe Deep Reinforcement Learningcited by 15
  13. 2023Policy Synthesis and Reinforcement Learning for Discounted LTLcited by 14
  14. 2023POMDP inference and robust solution via deep reinforcement learning: An application to railway optimal maintenancecited by 13
  15. 2024Reinforcement Learning with LTL and ω-Regular Objectives via Optimality-Preserving Translation to Average Rewardscited by 13
  16. 2020Optimal Inspection and Maintenance Planning for Deteriorating Structural Components through Dynamic Bayesian Networks and Markov Decision Processescited by 12
  17. 2022Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learningcited by 11
  18. 2021Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Modelcited by 9
  19. 2023SMARLA: A Safety Monitoring Approach for Deep Reinforcement Learning Agentscited by 9
  20. 2024Safety-Oriented Pruning and Interpretation of Reinforcement Learning Policiescited by 9
  21. 2020Runtime Safety Assurance Using Reinforcement Learningcited by 8
  22. 2024SAFE-RL: Saliency-Aware Counterfactual Explainer for Deep Reinforcement Learning Policiescited by 8
  23. 2024Verified Safe Reinforcement Learning for Neural Network Dynamic Modelscited by 7
  24. 2022Verified Probabilistic Policies for Deep Reinforcement Learningcited by 6
  25. 2024Enhancing RL Safety with Counterfactual LLM Reasoningcited by 6
  26. 2024Overcoming the Curse of Dimensionality in Reinforcement Learning Through Approximate Factorizationcited by 6
  27. 2022Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checkingcited by 5
  28. 2024Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure managementcited by 5
  29. 2020Reward Shaping for Reinforcement Learning with Omega-Regular Objectivescited by 4
  30. 2021Learning on Abstract Domains: A New Approach for Verifiable Guarantee in Reinforcement Learningcited by 3
  31. 2021Dependability Analysis of Deep Reinforcement Learning based Robotics and Autonomous Systems through Probabilistic Model Checkingcited by 3
  32. 2021A deep reinforcement learning model for predictive maintenance planning of road assets: Integrating LCA and LCCAcited by 3
  33. 2025Turn-based Multi-Agent Reinforcement Learning Model Checkingcited by 3
  34. 2026Formally Verifying and Explaining Sepsis Treatment Policies with COOL-MCcited by 3
  35. 2023Runtime Verification of Learning Properties for Reinforcement Learning Algorithmscited by 2
  36. 2025Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policiescited by 2
  37. 2021Towards Autonomous Pipeline Inspection with Hierarchical Reinforcement Learningcited by 1
  38. 2024Deep Multi-Objective Reinforcement Learning for Utility-Based Infrastructural Maintenance Optimizationcited by 1

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.