# Context-aware robot control using gesture episodes

arXiv: https://arxiv.org/abs/2301.09899 · HTML: https://mlsanity.com/paper/2301.09899/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2023.

## Built on by (most cited first, 1 of 1)

- [TransforMerger: Transformer-based Voice-Gesture Fusion for Robust Human-Robot Communication](https://mlsanity.com/paper/2504.01708/) (2025)

## Abstract (arXiv)

Collaborative robots became a popular tool for increasing productivity in partly automated manufacturing plants. Intuitive robot teaching methods are required to quickly and flexibly adapt the robot programs to new tasks. Gestures have an essential role in human communication. However, in human-robot-interaction scenarios, gesture-based user interfaces are so far used rarely, and if they employ a one-to-one mapping of gestures to robot control variables. In this paper, we propose a method that infers the user's intent based on gesture episodes, the context of the situation, and common sense. The approach is evaluated in a simulated table-top manipulation setting. We conduct deterministic experiments with simulated users and show that the system can even handle personal preferences of each user.