mlsanityData to 14 Oct 2026

Can you see how I learn? Human observers' inferences about Reinforcement Learning agents' learning processes

arXiv 2506.13583 ↗First posted 16 Jun 2025cs.HC cs.AI cs.ROcited by 0

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

Reinforcement Learning (RL) agents often exhibit learning behaviors that are not intuitively interpretable by human observers, which can result in suboptimal feedback in collaborative teaching settings. Yet, how humans perceive and interpret RL agent's learning behavior is largely unknown. In a bottom-up approach with two experiments, this work provides a data-driven understanding of the factors of human observers' understanding of the agent's learning process. A novel, observation-based paradigm to directly assess human inferences about agent learning was developed. In an exploratory interview study (textitN=9), we identify four core themes in human interpretations: Agent Goals, Knowledge, Decision Making, and Learning Mechanisms. A second confirmatory study (textitN=34) applied an expanded version of the paradigm across two tasks (navigation/manipulation) and two RL algorithms (tabular/function approximation). Analyses of 816 responses confirmed the reliability of the paradigm and refined the thematic framework, revealing how these themes evolve over time and interrelate. Our findings provide a human-centered understanding of how people make sense of agent learning, offering actionable insights for designing interpretable RL systems and improving transparency in Human-Robot Interaction.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 45 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

11 of 45 references.

References · 11

  1. 2016OpenAI Gymcited by 1,736
  2. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  3. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 158
  4. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 87
  5. 2021panda-gym: Open-source goal-conditioned environments for robotic learningcited by 59
  6. 2022Interactive Imitation Learning in Robotics: A Surveycited by 55
  7. 2021A Survey on Interactive Reinforcement Learning: Design Principles and Open Challengescited by 41
  8. 2017Agent-Agnostic Human-in-the-Loop Reinforcement Learningcited by 23
  9. 2023A Review of Communicating Robot Learning during Human-Robot Interactioncited by 2
  10. 2024"Give Me an Example Like This": Episodic Active Reinforcement Learning from Demonstrationscited by 2
  11. 2023Utility-based Adaptive Teaching Strategies using Bayesian Theory of Mindcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework
  2. 2025Objective Metrics for Human-Subjects Evaluation in Explainable Reinforcement Learning
  3. 2022Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback
  4. 2025A Theoretical Framework for Explaining Reinforcement Learning with Shapley Values
  5. 2024Revealing the Learning Process in Reinforcement Learning Agents Through Attention-Oriented Metrics
  6. 2020Explaining Conditions for Reinforcement Learning Behaviors from Real and Imagined Data
  7. 2024Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use
  8. 2026When Should an AI Act? A Human-Centered Model of Scene, Context, and Behavior for Agentic AI Design