# Enabling Robots to Communicate their Objectives

arXiv: https://arxiv.org/abs/1702.03465 · HTML: https://mlsanity.com/paper/1702.03465/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2017.

## Built on by (most cited first, 30 of 35)

- [Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations](https://mlsanity.com/paper/2210.11584/) (2022)
- [Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning](https://mlsanity.com/paper/2101.12446/) (2021)
- [Explainable Deep Reinforcement Learning: State of the Art and Challenges](https://mlsanity.com/paper/2301.09937/) (2023)
- [Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applications](https://mlsanity.com/paper/1805.07687/) (2018)
- [Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey](https://mlsanity.com/paper/2108.09003/) (2021)
- [Explainable Autonomous Robots: A Survey and Perspective](https://mlsanity.com/paper/2105.02658/) (2021)
- [Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomes](https://mlsanity.com/paper/2312.11118/) (2023)
- [Towards Modeling and Influencing the Dynamics of Human Learning](https://mlsanity.com/paper/2301.00901/) (2023)
- [Integrating Policy Summaries with Reward Decomposition for Explaining Reinforcement Learning Agents](https://mlsanity.com/paper/2210.11825/) (2022)
- ["I Don't Think So": Summarizing Policy Disagreements for Agent Comparison](https://mlsanity.com/paper/2102.03064/) (2021)
- [Reinforcement Learning in Practice: Opportunities and Challenges](https://mlsanity.com/paper/2202.11296/) (2022)
- [Reasoning about Counterfactuals to Improve Human Inverse Reinforcement Learning](https://mlsanity.com/paper/2203.01855/) (2022)
- [Contrastive Explanations for Comparing Preferences of Reinforcement Learning Agents](https://mlsanity.com/paper/2112.09462/) (2021)
- [Explaining Reward Functions to Humans for Better Human-Robot Collaboration](https://mlsanity.com/paper/2110.04192/) (2021)
- [An Information Bottleneck Characterization of the Understanding-Workload Tradeoff](https://mlsanity.com/paper/2310.07802/) (2023)
- [Summarising and Comparing Agent Dynamics with Contrastive Spatiotemporal Abstraction](https://mlsanity.com/paper/2201.07749/) (2022)
- [Deceptive Planning for Resource Allocation](https://mlsanity.com/paper/2206.01306/) (2022)
- [A Framework for Understanding and Visualizing Strategies of RL Agents](https://mlsanity.com/paper/2208.08552/) (2022)
- ["Guess what I'm doing": Extending legibility to sequential decision tasks](https://mlsanity.com/paper/2209.09141/) (2022)
- [Multi-Agent Strategy Explanations for Human-Robot Collaboration](https://mlsanity.com/paper/2311.11955/) (2023)
- [REACT: Revealing Evolutionary Action Consequence Trajectories for Interpretable Reinforcement Learning](https://mlsanity.com/paper/2404.03359/) (2024)
- [Interactive Explanations for Reinforcement-Learning Agents](https://mlsanity.com/paper/2504.05393/) (2025)
- [(When) Are Contrastive Explanations of Reinforcement Learning Helpful?](https://mlsanity.com/paper/2211.07719/) (2022)
- [I-CEE: Tailoring Explanations of Image Classification Models to User Expertise](https://mlsanity.com/paper/2312.12102/) (2023)
- [Explaining Learned Reward Functions with Counterfactual Trajectories](https://mlsanity.com/paper/2402.04856/) (2024)
- [Surrogate Fitness Metrics for Interpretable Reinforcement Learning](https://mlsanity.com/paper/2504.14645/) (2025)
- [BXRL: Behavior-Explainable Reinforcement Learning](https://mlsanity.com/paper/2603.23738/) (2026)
- [Global and Local Analysis of Interestingness for Competency-Aware Deep Reinforcement Learning](https://mlsanity.com/paper/2211.06376/) (2022)
- [ASQ-IT: Interactive Explanations for Reinforcement-Learning Agents](https://mlsanity.com/paper/2301.09941/) (2023)
- [Conveying Autonomous Robot Capabilities through Contrasting Behaviour Summaries](https://mlsanity.com/paper/2304.00367/) (2023)

## Abstract (arXiv)

The overarching goal of this work is to efficiently enable end-users to correctly anticipate a robot's behavior in novel situations. Since a robot's behavior is often a direct result of its underlying objective function, our insight is that end-users need to have an accurate mental model of this objective function in order to understand and predict what the robot will do. While people naturally develop such a mental model over time through observing the robot act, this familiarization process may be lengthy. Our approach reduces this time by having the robot model how people infer objectives from observed behavior, and then it selects those behaviors that are maximally informative. The problem of computing a posterior over objectives from observed behavior is known as Inverse Reinforcement Learning (IRL), and has been applied to robots learning human objectives. We consider the problem where the roles of human and robot are swapped. Our main contribution is to recognize that unlike robots, humans will not be exact in their IRL inference. We thus introduce two factors to define candidate approximate-inference models for human learning in this setting, and analyze them in a user study in the autonomous driving domain. We show that certain approximate-inference models lead to the robot generating example behaviors that better enable users to anticipate what it will do in novel situations. Our results also suggest, however, that additional research is needed in modeling how humans extrapolate from examples of robot behavior.