mlsanityData to 7 Oct 2026

Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations

arXiv 2002.01080 ↗First posted 4 Feb 2020cs.AIcited by 32

Not on the map. Why?

Abstract (arXiv)

As increasingly complex AI systems are introduced into our daily lives, it becomes important for such systems to be capable of explaining the rationale for their decisions and allowing users to contest these decisions. A significant hurdle to allowing for such explanatory dialogue could be the vocabulary mismatch between the user and the AI system. This paper introduces methods for providing contrastive explanations in terms of user-specified concepts for sequential decision-making settings where the system's model of the task may be best represented as an inscrutable model. We do this by building partial symbolic models of a local approximation of the task that can be leveraged to answer the user queries. We test these methods on a popular Atari game (Montezuma's Revenge) and variants of Sokoban (a well-known planning benchmark) and report the results of user studies to evaluate whether people find explanations generated in this form useful.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 9; 2024: 8; 2025: 5; 2026: 2.

32 papers.

Sort:

By area:

  1. 2022Logic-Based Explainability in Machine Learningcited by 47
  2. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 41
  3. 2021Symbols as a Lingua Franca for Bridging Human-AI Chasm for Explainable and Advisable AI Systemscited by 36
  4. 2024Theory of Mind abilities of Large Language Models in Human-Robot Interaction : An Illusion?cited by 22
  5. 2023State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understandingcited by 17
  6. 2024Natural Language Reinforcement Learningcited by 17
  7. 2021Discovering User-Interpretable Capabilities of Black-Box Planning Agentscited by 10
  8. 2023Explainable Multi-Agent Reinforcement Learning for Temporal Queriescited by 9
  9. 2022Symbol Guided Hindsight Priors for Reward Learning from Human Preferencescited by 8
  10. 2022Relative Behavioral Attributes: Filling the Gap between Symbolic Goal Specification and Reward Learning from Human Preferencescited by 7
  11. 2023Autonomous Capability Assessment of Sequential Decision-Making Systems in Stochastic Settings (Extended Version)cited by 6
  12. 2024Enabling MCTS Explainability for Sequential Planning Through Computation Tree Logiccited by 6
20 more
  1. 2022Experiential Explanations for Reinforcement Learningcited by 5
  2. 2022Towards customizable reinforcement learning agents: Enabling preference specification through online vocabulary expansioncited by 4
  3. 2023Exploiting Unlabeled Data for Feedback Efficient Human Preference based Reinforcement Learningcited by 4
  4. 2024Natural Language Reinforcement Learningcited by 4
  5. 2022A Mental-Model Centric Landscape of Human-AI Symbiosiscited by 3
  6. 2023Data Driven Reward Initialization for Preference based Reinforcement Learningcited by 3
  7. 2024ADESSE: Advice Explanations in Complex Repeated Decision-Making Environmentscited by 3
  8. 2025Nonparametric Identification of Latent Conceptscited by 3
  9. 2023A State Augmentation based approach to Reinforcement Learning from Human Preferencescited by 2
  10. 2025Explainable AI Systems Must Be Contestable: Here's How to Make It Happencited by 2
  11. 2023Goal Alignment: A Human-Aware Account of Value Alignment Problemcited by 1
  12. 2024Contrastive Explainable Clustering with Differential Privacycited by 1
  13. 2025Explainable Reinforcement Learning Agents Using World Modelscited by 1
  14. 2025On the Role of Domain Experts in Creating Effective Tutoring Systemscited by 1
  15. 2026BXRL: Behavior-Explainable Reinforcement Learningcited by 1
  16. 2023Methods and Mechanisms for Interactive Novelty Handling in Adversarial Environmentscited by 0
  17. 2024Explainable Human-AI Interaction: A Planning Perspectivecited by 0
  18. 2024Towards Probabilistic Planning of Explanations for Robot Navigationcited by 0
  19. 2025Explaining Decentralized Multi-Agent Reinforcement Learning Policiescited by 0
  20. 2026Toward Template-Free Explainability for Monte Carlo Tree Searchcited by 0