# Modeling Human Response to Robot Errors for Timely Error Detection

arXiv: https://arxiv.org/abs/2208.00565 · HTML: https://mlsanity.com/paper/2208.00565/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2022.

## Built on by (most cited first, 7 of 7)

- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [Predicting Human Perceptions of Robot Performance During Navigation Tasks](https://mlsanity.com/paper/2310.11590/) (2023)
- [Why Robots Are Bad at Detecting Their Mistakes: Limitations of Miscommunication Detection in Human-Robot Dialogue](https://mlsanity.com/paper/2506.20268/) (2025)
- [ERR@HRI 2.0 Challenge: Multimodal Detection of Errors and Failures in Human-Robot Conversations](https://mlsanity.com/paper/2507.13468/) (2025)
- [Adapting Robot's Explanation for Failures Based on Observed Human Behavior in Human-Robot Collaboration](https://mlsanity.com/paper/2504.09717/) (2025)
- [Training Models to Detect Successive Robot Errors from Human Reactions](https://mlsanity.com/paper/2510.09080/) (2025)
- [QuickLAP: Quick Language-Action Preference Learning for Semi-Autonomous Agents](https://mlsanity.com/paper/2511.17855/) (2025)

## Abstract (arXiv)

In human-robot collaboration, robot errors are inevitable -- damaging user trust, willingness to work together, and task performance. Prior work has shown that people naturally respond to robot errors socially and that in social interactions it is possible to use human responses to detect errors. However, there is little exploration in the domain of non-social, physical human-robot collaboration such as assembly and tool retrieval. In this work, we investigate how people's organic, social responses to robot errors may be used to enable timely automatic detection of errors in physical human-robot interactions. We conducted a data collection study to obtain facial responses to train a real-time detection algorithm and a case study to explore the generalizability of our method with different task settings and errors. Our results show that natural social responses are effective signals for timely detection and localization of robot errors even in non-social contexts and that our method is robust across a variety of task contexts, robot errors, and user responses. This work contributes to robust error detection without detailed task specifications.