mlsanityData to 14 Oct 2026

STACHE: Local Black-Box Explanations for Reinforcement Learning Policies

arXiv 2512.09909 ↗First posted 10 Dec 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

Reinforcement learning agents often behave unexpectedly in sparse-reward or safety-critical environments, creating a strong need for reliable debugging and verification tools. In this paper, we propose STACHE, a comprehensive framework for generating local, black-box explanations for an agent's specific action within discrete Markov games. Our method produces a Composite Explanation consisting of two complementary components: (1) a Robustness Region, the connected neighborhood of states where the agent's action remains invariant, and (2) Minimal Counterfactuals, the smallest state perturbations required to alter that decision. By exploiting the structure of factored state spaces, we introduce an exact, search-based algorithm that circumvents the fidelity gaps of surrogate models. Empirical validation on Gymnasium environments demonstrates that our framework not only explains policy actions, but also effectively captures the evolution of policy logic during training - from erratic, unstable behavior to optimized, robust strategies - providing actionable insights into agent sensitivity and decision boundaries.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 19 references that are in the map. First posted on arXiv 10 Dec 2025.

Builds on

14 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025A Survey on Explainable Deep Reinforcement Learning8 Feb

Other references · 13

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  3. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  4. 2017Robust Adversarial Reinforcement Learningcited by 451
  5. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
  6. 2018Programmatically Interpretable Reinforcement Learningcited by 193
  7. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 177
  8. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  9. 2017Visualizing and Understanding Atari Agentscited by 141
  10. 2019Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learningcited by 60
  11. 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 39
  12. 2022A Survey of Explainable Reinforcement Learningcited by 31
1 more
  1. 2023Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomescited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents
  2. 2025Explainable Reinforcement Learning via Temporal Policy Decomposition
  3. 2024Probabilistic Model Checking of Stochastic Reinforcement Learning Policies
  4. 2024XRL-Bench: A Benchmark for Evaluating and Comparing Explainable Reinforcement Learning Techniques
  5. 2026From Black Box to Executable Logic: Explainable Reinforcement Learning through Prolog Expert Systems
  6. 2025Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actions
  7. 2026Stochastic Actor-Critic: Mitigating Overestimation via Temporal Aleatoric Uncertainty
  8. 2025Model-Based Reinforcement Learning Under Confounding