mlsanityData to 14 Oct 2026

Exploring Computational User Models for Agent Policy Summarization

arXiv 1905.13271 ↗First posted 30 May 2019cs.LG stat.MLcited by 29

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

AI agents are being developed to support high stakes decision-making processes from driving cars to prescribing drugs, making it increasingly important for human users to understand their behavior. Policy summarization methods aim to convey strengths and weaknesses of such agents by demonstrating their behavior in a subset of informative states. Some policy summarization methods extract a summary that optimizes the ability to reconstruct the agent's policy under the assumption that users will deploy inverse reinforcement learning. In this paper, we explore the use of different models for extracting summaries. We introduce an imitation learning-based approach to policy summarization; we demonstrate through computational simulations that a mismatch between the model used to extract a summary and the model used to reconstruct the policy results in worse reconstruction quality; and we demonstrate through a human-subject study that people use different models to reconstruct policies in different contexts, and that matching the summary extraction model to these can improve performance. Together, our results suggest that it is important to carefully consider user models in policy summarization.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 30 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 7; 2022: 9; 2023: 6; 2024: 4; 2025: 3.

29 papers.

Sort:

By area:

  1. 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
  2. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 85
  3. 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 39
  4. 2022A Survey of Explainable Reinforcement Learningcited by 31
  5. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  6. 2023Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomescited by 14
  7. 2022Integrating Policy Summaries with Reward Decomposition for Explaining Reinforcement Learning Agentscited by 10
  8. 2021"I Don't Think So": Summarizing Policy Disagreements for Agent Comparisoncited by 6
  9. 2021Feature-Based Interpretable Reinforcement Learning based on State-Transition Modelscited by 6
  10. 2022Reasoning about Counterfactuals to Improve Human Inverse Reinforcement Learningcited by 6
  11. 2022Local Explanations for Reinforcement Learningcited by 5
  12. 2021Natural Language Specification of Reinforcement Learning Policies through Differentiable Decision Treescited by 4
17 more
  1. 2021Explaining Reward Functions to Humans for Better Human-Robot Collaborationcited by 3
  2. 2023Advantage Actor-Critic with Reasoner: Explaining the Agent's Behavior from an Exploratory Perspectivecited by 3
  3. 2022Towards Interpretable Deep Reinforcement Learning Models via Inverse Reinforcement Learningcited by 2
  4. 2022A Framework for Understanding and Visualizing Strategies of RL Agentscited by 2
  5. 2023Towards Reconciling Usability and Usefulness of Explainable AI Methodologiescited by 2
  6. 2024REACT: Revealing Evolutionary Action Consequence Trajectories for Interpretable Reinforcement Learningcited by 2
  7. 2024Navigating Trade-offs: Policy Summarization for Multi-Objective Reinforcement Learningcited by 2
  8. 2025Interactive Explanations for Reinforcement-Learning Agentscited by 2
  9. 2022(When) Are Contrastive Explanations of Reinforcement Learning Helpful?cited by 1
  10. 2023I-CEE: Tailoring Explanations of Image Classification Models to User Expertisecited by 1
  11. 2024Structure and Reduction of MCTS for Explainable-AIcited by 1
  12. 2025Surrogate Fitness Metrics for Interpretable Reinforcement Learningcited by 1
  13. 2022Global and Local Analysis of Interestingness for Competency-Aware Deep Reinforcement Learningcited by 0
  14. 2023ASQ-IT: Interactive Explanations for Reinforcement-Learning Agentscited by 0
  15. 2023IxDRL: A Novel Explainable Deep Reinforcement Learning Toolkit based on Analyses of Interestingnesscited by 0
  16. 2024Closed-loop Teaching via Demonstrations to Improve Policy Transparencycited by 0
  17. 2025"Trust me on this" Explaining Agent Behavior to a Human Terminatorcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Actions to Words: Towards Abstractive-Textual Policy Summarization in RL
  2. 2025Learning to summarize user information for personalized reinforcement learning from human feedback
  3. 2026InterPReT: Interactive Policy Restructuring and Training Enable Effective Imitation Learning from Laypersons
  4. 2017Inverse Reinforcement Learning from Summary Data
  5. 2024RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMs
  6. 2018Imitating Latent Policies from Observation
  7. 2020Learning to summarize from human feedback
  8. 2026When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning