# Reducing Human-Robot Goal State Divergence with Environment Design

arXiv: https://arxiv.org/abs/2404.15184 · HTML: https://mlsanity.com/paper/2404.15184/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 10 Apr 2024.

## Builds on (most cited first, 10 of 10)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [AI Safety Gridworlds](https://mlsanity.com/paper/1711.09883/) (2017)
- [The LAMA Planner: Guiding Cost-Based Anytime Planning with Landmarks](https://mlsanity.com/paper/1401.3839/) (2014)
- [Plan Explicability and Predictability for Robot Task Planning](https://mlsanity.com/paper/1511.08158/) (2015)
- [Model-Free Model Reconciliation](https://mlsanity.com/paper/1903.07198/) (2019)
- [Designing Environments Conducive to Interpretable Robot Behavior](https://mlsanity.com/paper/2007.00820/) (2020)
- [Not all users are the same: Providing personalized explanations for sequential decision making problems](https://mlsanity.com/paper/2106.12207/) (2021)
- [MACQ: A Holistic View of Model Acquisition Techniques](https://mlsanity.com/paper/2206.06530/) (2022)
- [Mitigating Negative Side Effects via Environment Shaping](https://mlsanity.com/paper/2102.07017/) (2021)

## Built on by (most cited first, 1 of 1)

- [Courteous Anticipation: Improving Long-Lived Task Planning in Persistent Shared Environments](https://mlsanity.com/paper/2607.20289/) (2026)

## Abstract (arXiv)

One of the most difficult challenges in creating successful human-AI collaborations is aligning a robot's behavior with a human user's expectations. When this fails to occur, a robot may misinterpret their specified goals, prompting it to perform actions with unanticipated, potentially dangerous side effects. To avoid this, we propose a new metric we call Goal State Divergence (GSD), which represents the difference between a robot's final goal state and the one a human user expected. In cases where GSD cannot be directly calculated, we show how it can be approximated using maximal and minimal bounds. We then input the GSD value into our novel human-robot goal alignment (HRGA) design problem, which identifies a minimal set of environment modifications that can prevent mismatches like this. To show the effectiveness of GSD for reducing differences between human-robot goal states, we empirically evaluate our approach on several standard benchmarks.