mlsanityData to 14 Oct 2026

SPOTting the Future: Lookahead Explanations for Deep Reinforcement Learning

arXiv 2608.09967 ↗First posted 28 Jul 2026cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

Deep reinforcement learning (DRL) agents achieve strong performance in complex environments, yet their decision-making processes remain difficult to interpret. We introduce SPOT (Sampling Policy Observation Tree), a novel model-agnostic, sampling-based framework for interpreting DRL policies. Given access to the policy and an environment simulator, SPOT constructs an interpretable finite-horizon tree by sampling actions and recursively simulating the resulting successor states. The tree provides an empirical representation of the policy's action preferences and their possible downstream evolution. We provide formal guarantees establishing SPOT's asymptotic recovery of the policy's unique most probable action and characterizing its disagreement behavior under high-entropy policies. We demonstrate SPOT in the SUMO-RL traffic-signal control domain. The case study illustrates how its tree-based representation can be used to inspect policy preferences, compare alternative future trajectories, and reveal downstream behaviors that are not visible through single-timestep feature-attribution methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 19 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

10 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025A Survey on Explainable Deep Reinforcement Learning8 Feb

Other references · 9

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  6. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  7. 2023FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcarecited by 60
  8. 2022Explainable Reinforcement Learning via Model Transformscited by 11
  9. 2022Lazy-MDPs: Towards Interpretable Reinforcement Learning by Learning When to Actcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SPOT: Scalable Policy Optimization with Trees for Markov Decision Processes
  2. 2018Verifiable Reinforcement Learning via Policy Extraction
  3. 2025Policy Gradient with Tree Search: Avoiding Local Optimas through Lookahead
  4. 2024BET: Explaining Deep Reinforcement Learning through The Error-Prone Decisions
  5. 2025A Practical Introduction to Deep Reinforcement Learning
  6. 2024RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Space
  7. 2025Interpret Policies in Deep Reinforcement Learning using SILVER with RL-Guided Labeling: A Model-level Approach to High-dimensional and Multi-action Environments
  8. 2024Optimizing Interpretable Decision Tree Policies for Reinforcement Learning