# Safe Explicable Planning

arXiv: https://arxiv.org/abs/2304.03773 · HTML: https://mlsanity.com/paper/2304.03773/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 44 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 10 of 10)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Reward learning from human preferences and demonstrations in Atari](https://mlsanity.com/paper/1811.06521/) (2018)
- [A Survey of Multi-Objective Sequential Decision-Making](https://mlsanity.com/paper/1402.0590/) (2014)
- [Near Optimal Behavior via Approximate State Abstraction](https://mlsanity.com/paper/1701.04113/) (2017)
- [Explainable Planning](https://mlsanity.com/paper/1709.10256/) (2017)
- [Explicability? Legibility? Predictability? Transparency? Privacy? Security? The Emerging Landscape of Interpretable Agent Behavior](https://mlsanity.com/paper/1811.09722/) (2018)
- [Plan Explicability and Predictability for Robot Task Planning](https://mlsanity.com/paper/1511.08158/) (2015)
- [Online Risk-Bounded Motion Planning for Autonomous Vehicles in Dynamic Environments](https://mlsanity.com/paper/1904.02341/) (2019)
- [Explicablility as Minimizing Distance from Expected Behavior](https://mlsanity.com/paper/1611.05497/) (2016)
- [Generating Active Explicable Plans in Human-Robot Teaming](https://mlsanity.com/paper/2109.08834/) (2021)

## Built on by (most cited first, 1 of 1)

- [Safe Explicable Policy Search](https://mlsanity.com/paper/2503.07848/) (2025)

## Abstract (arXiv)

Human expectations arise from their understanding of others and the world. In the context of human-AI interaction, this understanding may not align with reality, leading to the AI agent failing to meet expectations and compromising team performance. Explicable planning, introduced as a method to bridge this gap, aims to reconcile human expectations with the agent's optimal behavior, facilitating interpretable decision-making. However, an unresolved critical issue is ensuring safety in explicable planning, as it could result in explicable behaviors that are unsafe. To address this, we propose Safe Explicable Planning (SEP), which extends the prior work to support the specification of a safety bound. The goal of SEP is to find behaviors that align with human expectations while adhering to the specified safety criterion. Our approach generalizes the consideration of multiple objectives stemming from multiple models rather than a single model, yielding a Pareto set of safe explicable policies. We present both an exact method, guaranteeing finding the Pareto set, and a more efficient greedy method that finds one of the policies in the Pareto set. Additionally, we offer approximate solutions based on state aggregation to improve scalability. We provide formal proofs that validate the desired theoretical properties of these methods. Evaluation through simulations and physical robot experiments confirms the effectiveness of our approach for safe explicable planning.