# Predicting Human Perceptions of Robot Performance During Navigation Tasks

arXiv: https://arxiv.org/abs/2310.11590 · HTML: https://mlsanity.com/paper/2310.11590/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 98 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [DeepHuman: 3D Human Reconstruction from a Single Image](https://mlsanity.com/paper/1903.06473/) (2019)
- [Interactive Learning from Policy-Dependent Human Feedback](https://mlsanity.com/paper/1701.06049/) (2017)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Core Challenges of Social Robot Navigation: A Survey](https://mlsanity.com/paper/2103.05668/) (2021)
- [Sim-to-Real Transfer for Vision-and-Language Navigation](https://mlsanity.com/paper/2011.03807/) (2020)
- [Principles and Guidelines for Evaluating Social Robot Navigation Algorithms](https://mlsanity.com/paper/2306.16740/) (2023)
- [The EMPATHIC Framework for Task Learning from Implicit Human Feedback](https://mlsanity.com/paper/2009.13649/) (2020)
- [APReL: A Library for Active Preference-based Reward Learning Algorithms](https://mlsanity.com/paper/2108.07259/) (2021)
- [Efficient and Trustworthy Social Navigation Via Explicit and Implicit Robot-Human Communication](https://mlsanity.com/paper/1810.11556/) (2018)
- [JRDB: A Dataset and Benchmark of Egocentric Robot Visual Perception of Humans in Built Environments](https://mlsanity.com/paper/1910.11792/) (2019)
- [A Protocol for Validating Social Navigation Policies](https://mlsanity.com/paper/2204.05443/) (2022)
- [A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies](https://mlsanity.com/paper/1810.04871/) (2018)
- [Perceived Safety in Physical Human Robot Interaction -- A Survey](https://mlsanity.com/paper/2105.14499/) (2021)
- [An Approach to Deploy Interactive Robotic Simulators on the Web for HRI Experiments: Results in Social Robot Navigation](https://mlsanity.com/paper/2012.12336/) (2020)
- [Modeling Human Response to Robot Errors for Timely Error Detection](https://mlsanity.com/paper/2208.00565/) (2022)
- [Do you feel safe with your robot? Factors Influencing Perceived Safety in Human-Robot Interaction based on Subjective and Objective Measures](https://mlsanity.com/paper/2106.05854/) (2021)
- [Robot Navigation in Risky, Crowded Environments: Understanding Human Preferences](https://mlsanity.com/paper/2303.08284/) (2023)
- [Testing Robot System Safety by creating Hazardous Human Worker Behavior in Simulation](https://mlsanity.com/paper/2111.14433/) (2021)

## Built on by (most cited first, 2 of 2)

- [The Social Context of Human-Robot Interactions](https://mlsanity.com/paper/2508.13982/) (2025)
- [Rethinking the Implications of Human Feedback for Preference Learning in Human-Robot Collaboration](https://mlsanity.com/paper/2609.13982/) (2026)

## Abstract (arXiv)

Understanding human perceptions of robot performance is crucial for designing socially intelligent robots that can adapt to human expectations. Current approaches often rely on surveys, which can disrupt ongoing human-robot interactions. As an alternative, we explore predicting people's perceptions of robot performance using non-verbal behavioral cues and machine learning techniques. We contribute the SEAN TOGETHER Dataset consisting of observations of an interaction between a person and a mobile robot in Virtual Reality, together with perceptions of robot performance provided by users on a 5-point scale. We then analyze how well humans and supervised learning techniques can predict perceived robot performance based on different observation types (like facial expression and spatial behavior features). Our results suggest that facial expressions alone provide useful information, but in the navigation scenarios that we considered, reasoning about spatial features in context is critical for the prediction task. Also, supervised learning techniques outperformed humans' predictions in most cases. Further, when predicting robot performance as a binary classification task on unseen users' data, the F1-Score of machine learning models more than doubled that of predictions on a 5-point scale. This suggested good generalization capabilities, particularly in identifying performance directionality over exact ratings. Based on these findings, we conducted a real-world demonstration where a mobile robot uses a machine learning model to predict how a human who follows it perceives it. Finally, we discuss the implications of our results for implementing these supervised learning models in real-world navigation. Our work paves the path to automatically enhancing robot behavior based on observations of users and inferences about their perceptions of a robot.