# Learning to Communicate Functional States with Nonverbal Expressions for Improved Human-Robot Collaboration

arXiv: https://arxiv.org/abs/2404.19253 · HTML: https://mlsanity.com/paper/2404.19253/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 34 references that are in the map.
- First posted on arXiv 30 Apr 2024.

## Builds on (most cited first, 6 of 6)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Algorithms for multi-armed bandit problems](https://mlsanity.com/paper/1402.6028/) (2014)
- [Learning Reward Functions from Scale Feedback](https://mlsanity.com/paper/2110.00284/) (2021)
- [Reinforcement Learning Approaches in Social Robotics](https://mlsanity.com/paper/2009.09689/) (2020)
- [Human-centered collaborative robots with deep reinforcement learning](https://mlsanity.com/paper/2007.01009/) (2020)
- [Robot gains Social Intelligence through Multimodal Deep Reinforcement Learning](https://mlsanity.com/paper/1702.07492/) (2017)

## Abstract (arXiv)

Collaborative robots must effectively communicate their internal state to humans to enable a smooth interaction. Nonverbal communication is widely used to communicate information during human-robot interaction, however, such methods may also be misunderstood, leading to communication errors. In this work, we explore modulating the acoustic parameter values (pitch bend, beats per minute, beats per loop) of nonverbal auditory expressions to convey functional robot states (accomplished, progressing, stuck). We propose a reinforcement learning (RL) algorithm based on noisy human feedback to produce accurately interpreted nonverbal auditory expressions. The proposed approach was evaluated through a user study with 24 participants. The results demonstrate that: 1. Our proposed RL-based approach is able to learn suitable acoustic parameter values which improve the users' ability to correctly identify the state of the robot. 2. Algorithm initialization informed by previous user data can be used to significantly speed up the learning process. 3. The method used for algorithm initialization strongly influences whether participants converge to similar sounds for each robot state. 4. Modulation of pitch bend has the largest influence on user association between sounds and robotic states.