Behaviour Discovery and Attribution for Explainable Reinforcement Learning
On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL
Abstract (arXiv)
Building trust in reinforcement learning (RL) agents requires understanding why they make certain decisions, especially in high-stakes applications like robotics, healthcare, and finance. Existing explainability methods often focus on single states or entire trajectories, either providing only local, step-wise insights or attributing decisions to coarse, episodelevel summaries. Both approaches miss the recurring strategies and temporally extended patterns that actually drive agent behavior across multiple decisions. We address this gap by proposing a fully offline, reward-free framework for behavior discovery and segmentation, enabling the attribution of actions to meaningful and interpretable behavior segments that capture recurring patterns appearing across multiple trajectories. Our method identifies coherent behavior clusters from state-action sequences and attributes individual actions to these clusters for fine-grained, behavior-centric explanations. Evaluations on four diverse offline RL environments show that our approach discovers meaningful behaviors and outperforms trajectory-level baselines in fidelity, human preference, and cluster coherence. Our code is publicly available.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 39 of its 56 references that are in the map. First posted on arXiv 19 Mar 2025.
Builds on
39 of 56 references.
Closest ancestors · 2
Same direction, newest first
- 2025A Survey on Explainable Deep Reinforcement Learning8 Feb
- 2024RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation5 May
Other references · 37
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2017Neural Discrete Representation Learningcited by 4,712
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
- 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
- 2017A Brief Survey of Deep Reinforcement Learningcited by 483
- 2016The Option-Critic Architecturecited by 448
- 2020Deep Structural Causal Models for Tractable Counterfactual Inferencecited by 211
25 more
- 2018Programmatically Interpretable Reinforcement Learningcited by 193
- 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 182
- 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
- 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
- 2017Visualizing and Understanding Atari Agentscited by 141
- 2018Variational Option Discovery Algorithmscited by 122
- 2023Learning to Act without Actionscited by 121
- 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 110
- 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 71
- 2016Graying the black box: Understanding DQNscited by 68
- 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 67
- 2017When Waiting is not an Option : Learning Options with a Deliberation Costcited by 58
- 2019Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attributioncited by 50
- 2018Transparency and Explanation in Deep Reinforcement Learning Neural Networkscited by 40
- 2023Action-Quantized Offline Reinforcement Learning for Robotic Skill Learningcited by 36
- 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
- 2020Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Mapscited by 30
- 2023Explaining Reinforcement Learning with Shapley Valuescited by 26
- 2023Learning to Identify Critical States for Reinforcement Learning from Videoscited by 9
- 2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
- 2023Explaining RL Decisions with Trajectoriescited by 7
- 2022Bayesian Nonparametrics for Offline Skill Discoverycited by 5
- 2020Explainable Deep Reinforcement Learning for UAV Autonomous Navigationcited by 4
- 2022Behavior Estimation from Multi-Source Data for Offline Reinforcement Learningcited by 2
- 2023Offline Skill Graph (OSG): A Framework for Learning and Planning using Offline Reinforcement Learning Skillscited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024'Explaining RL Decisions with Trajectories': A Reproducibility Study
- 2025A Theoretical Framework for Explaining Reinforcement Learning with Shapley Values
- 2025Explainable Reinforcement Learning via Temporal Policy Decomposition
- 2025Know your Trajectory -- Trustworthy Reinforcement Learning deployment through Importance-Based Trajectory Analysis
- 2020Explaining Conditions for Reinforcement Learning Behaviors from Real and Imagined Data
- 2023A Closer Look at Reward Decomposition for High-Level Robotic Explanations
- 2025Objective Metrics for Human-Subjects Evaluation in Explainable Reinforcement Learning
- 2026From Features to Actions: Explainability in Traditional and Agentic AI Systems