mlsanityData to 14 Oct 2026

Counterfactual Explanations for Continuous Action Reinforcement Learning

arXiv 2505.12701 ↗First posted 19 May 2025cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

Reinforcement Learning (RL) has shown great promise in domains like healthcare and robotics but often struggles with adoption due to its lack of interpretability. Counterfactual explanations, which address "what if" scenarios, provide a promising avenue for understanding RL decisions but remain underexplored for continuous action spaces. We propose a novel approach for generating counterfactual explanations in continuous action RL by computing alternative action sequences that improve outcomes while minimizing deviations from the original sequence. Our approach leverages a distance metric for continuous actions and accounts for constraints such as adhering to predefined policies in specific states. Evaluations in two RL domains, Diabetes Control and Lunar Lander, demonstrate the effectiveness, efficiency, and generalization of our approach, enabling more interpretable and trustworthy RL applications.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 21 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

12 of 21 references.

References · 12

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2016OpenAI Gymcited by 1,736
  4. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  5. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  6. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  7. 2020Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Reviewcited by 174
  8. 2021Counterfactual Explanations in Sequential Decision Making Under Uncertaintycited by 37
  9. 2020Basal Glucose Control in Type 1 Diabetes using Deep Reinforcement Learning: An In Silico Validationcited by 18
  10. 2023Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomescited by 14
  11. 2022Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunitiescited by 13
  12. 2022Explaining Reinforcement Learning Policies through Counterfactual Trajectoriescited by 7

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Interpret Policies in Deep Reinforcement Learning using SILVER with RL-Guided Labeling: A Model-level Approach to High-dimensional and Multi-action Environmentscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024ACTER: Diverse and Actionable Counterfactual Sequences for Explaining and Diagnosing RL Policies
  2. 2026Explaining Reinforcement Learning Decisions in Self-adaptive Systems
  3. 2024Explaining Reinforcement Learning: A Counterfactual Shapley Values Approach
  4. 2026Class-Aware Reinforcement Learning for Counterfactual Explanation Generation
  5. 2025Explainable Reinforcement Learning via Temporal Policy Decomposition
  6. 2023Explaining RL Decisions with Trajectories
  7. 2024Explaining Learned Reward Functions with Counterfactual Trajectories
  8. 2022Decisions that Explain Themselves: A User-Centric Deep Reinforcement Learning Explanation System