# Body Gesture Recognition to Control a Social Robot

arXiv: https://arxiv.org/abs/2206.07538 · HTML: https://mlsanity.com/paper/2206.07538/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 15 Jun 2022.

## Builds on (most cited first, 7 of 7)

- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Survey on Emotional Body Gesture Recognition](https://mlsanity.com/paper/1801.07481/) (2018)
- [Sign Language Video Retrieval with Free-Form Textual Queries](https://mlsanity.com/paper/2201.02495/) (2022)
- [Improved Optical Flow for Gesture-based Human-robot Interaction](https://mlsanity.com/paper/1905.08685/) (2019)
- [A Proposed Set of Communicative Gestures for Human Robot Interaction and an RGB Image-based Gesture Recognizer Implemented in ROS](https://mlsanity.com/paper/2109.09908/) (2021)
- [Enabling Intuitive Human-Robot Teaming Using Augmented Reality and Gesture Control](https://mlsanity.com/paper/1909.06415/) (2019)
- [Gesture Recognition for Initiating Human-to-Robot Handovers](https://mlsanity.com/paper/2007.09945/) (2020)

## Abstract (arXiv)

In this work, we propose a gesture based language to allow humans to interact with robots using their body in a natural way. We have created a new gesture detection model using neural networks and a custom dataset of humans performing a set of body gestures to train our network. Furthermore, we compare body gesture communication with other communication channels to acknowledge the importance of adding this knowledge to robots. The presented approach is extensively validated in diverse simulations and real-life experiments with non-trained volunteers. This attains remarkable results and shows that it is a valuable framework for social robotics applications, such as human robot collaboration or human-robot interaction.