mlsanityData to 14 Oct 2026

"What are my options?": Explaining RL Agents with Diverse Near-Optimal Alternatives (Extended)

arXiv 2506.09901 ↗First posted 11 Jun 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

In this work, we provide an extended discussion of a new approach to explainable Reinforcement Learning called Diverse Near-Optimal Alternatives (DNA), first proposed at L4DC 2025. DNA seeks a set of reasonable "options" for trajectory-planning agents, optimizing policies to produce qualitatively diverse trajectories in Euclidean space. In the spirit of explainability, these distinct policies are used to "explain" an agent's options in terms of available trajectory shapes from which a human user may choose. In particular, DNA applies to value function-based policies on Markov decision processes where agents are limited to continuous trajectories. Here, we describe DNA, which uses reward shaping in local, modified Q-learning problems to solve for distinct policies with guaranteed epsilon-optimality. We show that it successfully returns qualitatively different policies that constitute meaningfully different "options" in simulation, including a brief comparison to related approaches in the stochastic optimization field of Quality Diversity. Beyond the explanatory motivation, this work opens new possibilities for exploration and adaptive planning in RL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 21 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

13 of 21 references.

References · 13

  1. 2015Illuminating search spaces by mapping elitescited by 425
  2. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  3. 2018Lipschitz Continuity in Model-based Reinforcement Learningcited by 91
  4. 2020Quality-Diversity Optimization: a novel branch of stochastic optimizationcited by 60
  5. 2018Establishing Appropriate Trust via Critical Statescited by 48
  6. 2020The Emerging Landscape of Explainable AI Planning and Decision Makingcited by 44
  7. 2019Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulationcited by 35
  8. 2022A Survey of Explainable Reinforcement Learningcited by 31
  9. 2023pyribs: A Bare-Bones Python Library for Quality Diversity Optimizationcited by 30
  10. 2023Uncertain Quality-Diversity: Evaluation methodology and new methods for Quality-Diversity in Uncertain Domainscited by 13
  11. 2022Explainable Reinforcement Learning via Model Transformscited by 11
  12. 2022Training Diverse High-Dimensional Controllers by Scaling Covariance Matrix Adaptation MAP-Annealingcited by 10
1 more
  1. 2023Pointwise-in-Time Explanation for Linear Temporal Logic Rulescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Trajectory First: A Curriculum for Discovering Diverse Policies
  2. 2024'Explaining RL Decisions with Trajectories': A Reproducibility Study
  3. 2025Balancing Interpretability and Performance in Reinforcement Learning: An Adaptive Spectral Based Linear Approach
  4. 2026Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning
  5. 2025Explainable Reinforcement Learning via Temporal Policy Decomposition
  6. 2025Statistical and Algorithmic Foundations of Reinforcement Learning
  7. 2024Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning
  8. 2025A View of the Certainty-Equivalence Method for PAC RL as an Application of the Trajectory Tree Method