# Hands-On Robotics: Enabling Communication Through Direct Gesture Control

arXiv: https://arxiv.org/abs/2401.09077 · HTML: https://mlsanity.com/paper/2401.09077/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2024.

## Built on by (most cited first, 1 of 1)

- [PoseLess: Depth-Free Vision-to-Joint Control via Direct Image Mapping with VLM](https://mlsanity.com/paper/2503.07111/) (2025)

## Abstract (arXiv)

Effective Human-Robot Interaction (HRI) is fundamental to seamlessly integrating robotic systems into our daily lives. However, current communication modes require additional technological interfaces, which can be cumbersome and indirect. This paper presents a novel approach, using direct motion-based communication by moving a robot's end effector. Our strategy enables users to communicate with a robot by using four distinct gestures -- two handshakes ('formal' and 'informal') and two letters ('W' and 'S'). As a proof-of-concept, we conducted a user study with 16 participants, capturing subjective experience ratings and objective data for training machine learning classifiers. Our findings show that the four different gestures performed by moving the robot's end effector can be distinguished with close to 100% accuracy. Our research offers implications for the design of future HRI interfaces, suggesting that motion-based interaction can empower human operators to communicate directly with robots, removing the necessity for additional hardware.