# To Whom are You Talking? A Deep Learning Model to Endow Social Robots with Addressee Estimation Skills

arXiv: https://arxiv.org/abs/2308.10757 · HTML: https://mlsanity.com/paper/2308.10757/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 31 references that are in the map.
- First posted on arXiv 21 Aug 2023.

## Builds on (most cited first, 3 of 3)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Bi-modal First Impressions Recognition using Temporally Ordered Deep Audio and Stochastic Visual Features](https://mlsanity.com/paper/1610.10048/) (2016)

## Built on by (most cited first, 2 of 2)

- [Real-time Addressee Estimation: Deployment of a Deep-Learning Model on the iCub Robot](https://mlsanity.com/paper/2311.05334/) (2023)
- [A Multi-Modal Explainability Approach for Human-Aware Robots in Multi-Party Conversation](https://mlsanity.com/paper/2407.03340/) (2024)

## Abstract (arXiv)

Communicating shapes our social word. For a robot to be considered social and being consequently integrated in our social environment it is fundamental to understand some of the dynamics that rule human-human communication. In this work, we tackle the problem of Addressee Estimation, the ability to understand an utterance's addressee, by interpreting and exploiting non-verbal bodily cues from the speaker. We do so by implementing an hybrid deep learning model composed of convolutional layers and LSTM cells taking as input images portraying the face of the speaker and 2D vectors of the speaker's body posture. Our implementation choices were guided by the aim to develop a model that could be deployed on social robots and be efficient in ecological scenarios. We demonstrate that our model is able to solve the Addressee Estimation problem in terms of addressee localisation in space, from a robot ego-centric point of view.