mlsanityData to 7 Oct 2026

"I Don't Think So": Summarizing Policy Disagreements for Agent Comparison

arXiv 2102.03064 ↗First posted 5 Feb 2021cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

With Artificial Intelligence on the rise, human interaction with autonomous agents becomes more frequent. Effective human-agent collaboration requires users to understand the agent's behavior, as failing to do so may cause reduced productivity, misuse or frustration. Agent strategy summarization methods are used to describe the strategy of an agent to its destined user through demonstration. A summary's objective is to maximize the user's understanding of the agent's aptitude by showcasing its behaviour in a selected set of world states. While shown to be useful, we show that current methods are limited when tasked with comparing between agents, as each summary is independently generated for a specific agent. In this paper, we propose a novel method for generating dependent and contrastive summaries that emphasize the differences between agent policies by identifying states in which the agents disagree on the best course of action. We conduct user studies to assess the usefulness of disagreement-based summaries for identifying superior agents and conveying agent differences. Results show disagreement-based summaries lead to improved user performance compared to summaries generated using HIGHLIGHTS, a strategy summarization algorithm which generates summaries for each agent independently.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 9 of its 20 references that are in the map. First posted on arXiv 5 Feb 2021.

Builds on

9 of 20 references.

References · 9

  1. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  2. 2018Metrics for Explainable AI: Challenges and Prospectscited by 257
  3. 2017Visualizing and Understanding Atari Agentscited by 141
  4. 2020Explainable Reinforcement Learning: A Surveycited by 139
  5. 2020Explainability in Deep Reinforcement Learningcited by 96
  6. 2019Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitationscited by 50
  7. 2017Enabling Robots to Communicate their Objectivescited by 34
  8. 2020Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Mapscited by 30
  9. 2019Exploring Computational User Models for Agent Policy Summarizationcited by 29

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 2; 2024: 1.

6 papers.

  1. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  2. 2023Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomescited by 13
  3. 2024Enhancing RL Safety with Counterfactual LLM Reasoningcited by 6
  4. 2022A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challengescited by 3
  5. 2022(When) Are Contrastive Explanations of Reinforcement Learning Helpful?cited by 1
  6. 2023Conveying Autonomous Robot Capabilities through Contrasting Behaviour Summariescited by 0