mlsanityData to 14 Oct 2026

Strategically Linked Decisions in Long-Term Planning and Reinforcement Learning

arXiv 2505.16833 ↗First posted 22 May 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Long-term planning, as in reinforcement learning (RL), involves finding strategies: actions that collectively work toward a goal rather than individually optimizing their immediate outcomes. As part of a strategy, some actions are taken at the expense of short-term benefit to enable future actions with even greater returns. These actions are only advantageous if followed up by the actions they facilitate, consequently, they would not have been taken if those follow-ups were not available. In this paper, we quantify such dependencies between planned actions with strategic link scores: the drop in the likelihood of one decision under the constraint that a follow-up decision is no longer available. We demonstrate the utility of strategic link scores through three practical applications: (i) explaining black-box RL agents by identifying strategically linked pairs among decisions they make, (ii) improving the worst-case performance of decision support systems by distinguishing whether recommended actions can be adopted as standalone improvements or whether they are strategically linked hence requiring a commitment to a broader strategy to be effective, and (iii) characterizing the planning processes of non-RL agents purely through interventions aimed at measuring strategic link scores - as an example, we consider a realistic traffic simulator and analyze through road closures the effective planning horizon of the emergent routing behavior of many drivers.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 28 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

21 of 28 references.

References · 21

  1. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  2. 2018Programmatically Interpretable Reinforcement Learningcited by 193
  3. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  4. 2017Safe Policy Improvement with Baseline Bootstrappingcited by 144
  5. 2017Visualizing and Understanding Atari Agentscited by 141
  6. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  7. 2022Supported Policy Optimization for Offline Reinforcement Learningcited by 74
  8. 2018Toward Interpretable Deep Reinforcement Learning with Linear Model U-Treescited by 65
  9. 2020Quantifying Differences in Reward Functionscited by 63
  10. 2017Interpretable Policies for Reinforcement Learning by Genetic Programmingcited by 48
  11. 2018Contrastive Explanations for Reinforcement Learning in terms of Expected Consequencescited by 44
  12. 2018Transparency and Explanation in Deep Reinforcement Learning Neural Networkscited by 40
9 more
  1. 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 39
  2. 2017Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanationscited by 37
  3. 2019Generation of Policy-Level Explanations for Reinforcement Learningcited by 37
  4. 2020What Did You Think Would Happen? Explaining Agent Behaviour Through Intended Outcomescited by 20
  5. 2020Explainable robotic systems: Understanding goal-driven actions in a reinforcement learning scenariocited by 19
  6. 2023GANterfactual-RL: Understanding Reinforcement Learning Agents' Strategies through Visual Counterfactual Explanationscited by 17
  7. 2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
  8. 2022Policy Optimization with Sparse Global Contrastive Explanationscited by 2
  9. 2024Decision-Point Guided Safe Policy Improvementcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Strict Subgoal Execution: Reliable Long-Horizon Planning in Hierarchical Reinforcement Learning
  2. 2026Relating Reinforcement Learning to Dynamic Programming-Based Planning
  3. 2023Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities
  4. 2024'Explaining RL Decisions with Trajectories': A Reproducibility Study
  5. 2026SUN: Reaching for Novelty in Reinforcement Learning
  6. 2025Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learning
  7. 2026Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis
  8. 2026Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RL