# One-shot Policy Elicitation via Semantic Reward Manipulation

arXiv: https://arxiv.org/abs/2101.01860 · HTML: https://mlsanity.com/paper/2101.01860/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 6 Jan 2021.

## Builds on (most cited first, 15 of 15)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Explaining Explanations in AI](https://mlsanity.com/paper/1811.01439/) (2018)
- [Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches](https://mlsanity.com/paper/1912.05100/) (2019)
- [An Evaluation of the Human-Interpretability of Explanation](https://mlsanity.com/paper/1902.00006/) (2019)
- [Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences](https://mlsanity.com/paper/1807.08706/) (2018)
- [Explainable Security](https://mlsanity.com/paper/1807.04178/) (2018)
- [Autonomous Self-Explanation of Behavior for Interactive Reinforcement Learning Agents](https://mlsanity.com/paper/1810.08811/) (2018)
- [AI Challenges in Human-Robot Cognitive Teaming](https://mlsanity.com/paper/1707.04775/) (2017)
- [Robot Planning with Mathematical Models of Human State and Action](https://mlsanity.com/paper/1705.04226/) (2017)
- [Plan Explanations as Model Reconciliation -- An Empirical Study](https://mlsanity.com/paper/1802.01013/) (2018)
- [Learning What Information to Give in Partially Observed Domains](https://mlsanity.com/paper/1805.08263/) (2018)
- [Automated Failure-Mode Clustering and Labeling for Informed Car-To-Driver Handover in Autonomous Vehicles](https://mlsanity.com/paper/2005.04439/) (2020)

## Built on by (most cited first, 1 of 1)

- [Structured State Reconciliation for Human-AI Task Handover](https://mlsanity.com/paper/2608.28907/) (2026)

## Abstract (arXiv)

Synchronizing expectations and knowledge about the state of the world is an essential capability for effective collaboration. For robots to effectively collaborate with humans and other autonomous agents, it is critical that they be able to generate intelligible explanations to reconcile differences between their understanding of the world and that of their collaborators. In this work we present Single-shot Policy Explanation for Augmenting Rewards (SPEAR), a novel sequential optimization algorithm that uses semantic explanations derived from combinations of planning predicates to augment agents' reward functions, driving their policies to exhibit more optimal behavior. We provide an experimental validation of our algorithm's policy manipulation capabilities in two practically grounded applications and conclude with a performance analysis of SPEAR on domains of increasingly complex state space and predicate counts. We demonstrate that our method makes substantial improvements over the state-of-the-art in terms of runtime and addressable problem size, enabling an agent to leverage its own expertise to communicate actionable information to improve another's performance.