# Using Surprise Index for Competency Assessment in Autonomous Decision-Making

arXiv: https://arxiv.org/abs/2312.09033 · HTML: https://mlsanity.com/paper/2312.09033/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 12 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Builds on (most cited first, 4 of 4)

- [Conflict and Surprise: Heuristics for Model Revision](https://mlsanity.com/paper/1303.5728/) (2013)
- [Analysis in HUGIN of Data Conflict](https://mlsanity.com/paper/1304.1146/) (2013)
- [A Factor-Based Framework for Decision-Making Competency Self-Assessment](https://mlsanity.com/paper/2203.11981/) (2022)
- [Dynamic Competency Self-Assessment for Autonomous Agents](https://mlsanity.com/paper/2303.01646/) (2023)

## Abstract (arXiv)

This paper considers the problem of evaluating an autonomous system's competency in performing a task, particularly when working in dynamic and uncertain environments. The inherent opacity of machine learning models, from the perspective of the user, often described as a `black box', poses a challenge. To overcome this, we propose using a measure called the Surprise index, which leverages available measurement data to quantify whether the dynamic system performs as expected. We show that the surprise index can be computed in closed form for dynamic systems when observed evidence in a probabilistic model if the joint distribution for that evidence follows a multivariate Gaussian marginal distribution. We then apply it to a nonlinear spacecraft maneuver problem, where actions are chosen by a reinforcement learning agent and show it can indicate how well the trajectory follows the required orbit.