mlsanityData to 7 Oct 2026

Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policies

arXiv 2501.03142 ↗First posted 6 Jan 2025cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (RL) policies can demonstrate unsafe behaviors and are challenging to interpret. To address these challenges, we combine RL policy model checking--a technique for determining whether RL policies exhibit unsafe behaviors--with co-activation graph analysis--a method that maps neural network inner workings by analyzing neuron activation patterns--to gain insight into the safe RL policy's sequential decision-making. This combination lets us interpret the RL policy's inner workings for safe decision-making. We demonstrate its applicability in various experiments.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 53 references that are in the map. First posted on arXiv 6 Jan 2025.

Builds on

17 of 53 references.

References · 17

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2018Metrics for Explainable AI: Challenges and Prospectscited by 257
  3. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  4. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 74
  5. 2020The Probabilistic Model Checker Stormcited by 64
  6. 2021Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)cited by 55
  7. 2018Establishing Appropriate Trust via Critical Statescited by 47
  8. 2019An Inductive Synthesis Framework for Verifiable Reinforcement Learningcited by 39
  9. 2014Verification of Markov Decision Processes using Learning Algorithmscited by 38
  10. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  11. 2022COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checkingcited by 19
  12. 2015Permissive Controller Synthesis for Probabilistic Systemscited by 16
5 more
  1. 2020Statistically Model Checking PCTL Specifications on Markov Decision Processes via Reinforcement Learningcited by 12
  2. 2021Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Modelcited by 9
  3. 2024Safety-Oriented Pruning and Interpretation of Reinforcement Learning Policiescited by 9
  4. 2023Fast unfolding of communities in large networks: 15 years latercited by 6
  5. 2024Enhancing RL Safety with Counterfactual LLM Reasoningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Verifying Memoryless Sequential Decision-making of Large Language Modelscited by 1
  2. 2026COOL-MC: Verifying and Explaining RL Policies for Multi-bridge Network Maintenancecited by 0