mlsanityData to 7 Oct 2026

COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checking

arXiv 2209.07133 ↗First posted 15 Sep 2022cs.LG cs.LOcited by 19

Not on the map. Why?

Abstract (arXiv)

This paper presents COOL-MC, a tool that integrates state-of-the-art reinforcement learning (RL) and model checking. Specifically, the tool builds upon the OpenAI gym and the probabilistic model checker Storm. COOL-MC provides the following features: (1) a simulator to train RL policies in the OpenAI gym for Markov decision processes (MDPs) that are defined as input for Storm, (2) a new model builder for Storm, which uses callback functions to verify (neural network) RL policies, (3) formal abstractions that relate models and policies specified in OpenAI gym or Storm, and (4) algorithms to obtain bounds on the performance of so-called permissive policies. We describe the components and architecture of COOL-MC and demonstrate its features on multiple benchmark environments.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 14 of its 40 references that are in the map. First posted on arXiv 15 Sep 2022.

Builds on

14 of 40 references.

References · 14

  1. 2016OpenAI Gymcited by 1,684
  2. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  3. 2017Safe Reinforcement Learning via Shieldingcited by 427
  4. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 74
  5. 2021Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)cited by 55
  6. 2014Verification of Markov Decision Processes using Learning Algorithmscited by 38
  7. 2017A storm is Coming: A Modern Probabilistic Model Checkercited by 34
  8. 2015Safety-Constrained Reinforcement Learning for MDPscited by 31
  9. 2015Permissive Controller Synthesis for Probabilistic Systemscited by 16
  10. 2020Enforcing Almost-Sure Reachability in POMDPscited by 13
  11. 2020Statistically Model Checking PCTL Specifications on Markov Decision Processes via Reinforcement Learningcited by 12
  12. 2022Specification-Guided Learning of Nash Equilibria with High Social Welfarecited by 8
2 more
  1. 2021Mungojerrie: Reinforcement Learning of Linear-Time Objectivescited by 6
  2. 2022Verified Probabilistic Policies for Deep Reinforcement Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 0; 2024: 4; 2025: 6; 2026: 7.

19 papers.

Sort:

By area:

  1. 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learningcited by 11
  2. 2024Safety-Oriented Pruning and Interpretation of Reinforcement Learning Policiescited by 9
  3. 2024Enhancing RL Safety with Counterfactual LLM Reasoningcited by 6
  4. 2022Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checkingcited by 5
  5. 2024Probabilistic Model Checking of Stochastic Reinforcement Learning Policiescited by 5
  6. 2025Turn-based Multi-Agent Reinforcement Learning Model Checkingcited by 3
  7. 2026Formally Verifying and Explaining Sepsis Treatment Policies with COOL-MCcited by 3
  8. 2024Application of AI to formal methods - an analysis of current trendscited by 2
  9. 2025Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policiescited by 2
  10. 2025Verifying Memoryless Sequential Decision-making of Large Language Modelscited by 1
  11. 2026Easy-to-Use Shielding for Reinforcement Learningcited by 1
  12. 2025Safe But Not Sorry: Reducing Over-Conservatism in Safety Critics via Uncertainty-Aware Modulationcited by 0
7 more
  1. 2025Formal Verification of Noisy Quantum Reinforcement Learning Policiescited by 0
  2. 2025Translating the Rashomon Effect to Sequential Decision-Making Taskscited by 0
  3. 2026COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Managementcited by 0
  4. 2026COOL-MC: Verifying and Explaining RL Policies for Multi-bridge Network Maintenancecited by 0
  5. 2026Formal Verification of Learned Multi-Agent Communication Policies via Decision Tree Distillationcited by 0
  6. 2026Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policiescited by 0
  7. 2026Automated Testing of LLM-Based Post Hoc Explainers Using Model Checking as an Oraclecited by 0