# Enabling a Social Robot to Process Social Cues to Detect when to Help a User

arXiv: https://arxiv.org/abs/2110.11075 · HTML: https://mlsanity.com/paper/2110.11075/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 44 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 6 of 6)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users](https://mlsanity.com/paper/1301.7385/) (2013)
- [Platform for Situated Intelligence](https://mlsanity.com/paper/2103.15975/) (2021)
- [A Knowledge Driven Approach to Adaptive Assistance Using Preference Reasoning and Explanation](https://mlsanity.com/paper/2012.02904/) (2020)
- [Supporting User Autonomy with Multimodal Fusion to Detect when a User Needs Assistance from a Social Robot](https://mlsanity.com/paper/2012.04078/) (2020)
- [Developing Computational Models of Social Assistance to Guide Socially Assistive Robots](https://mlsanity.com/paper/1909.06510/) (2019)

## Abstract (arXiv)

It is important for socially assistive robots to be able to recognize when a user needs and wants help. Such robots need to be able to recognize human needs in a real-time manner so that they can provide timely assistance. We propose an architecture that uses social cues to determine when a robot should provide assistance. Based on a multimodal fusion approach upon eye gaze and language modalities, our architecture is trained and evaluated on data collected in a robot-assisted Lego building task. By focusing on social cues, our architecture has minimal dependencies on the specifics of a given task, enabling it to be applied in many different contexts. Enabling a social robot to recognize a user's needs through social cues can help it to adapt to user behaviors and preferences, which in turn will lead to improved user experiences.