# Evaluating Pointing Gestures for Target Selection in Human-Robot Collaboration

arXiv: https://arxiv.org/abs/2506.22116 · HTML: https://mlsanity.com/paper/2506.22116/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 27 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Builds on (most cited first, 3 of 3)

- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [OpenDR: An Open Toolkit for Enabling High Performance, Low Footprint Deep Learning for Robotics](https://mlsanity.com/paper/2203.00403/) (2022)
- [Solving Visual Object Ambiguities when Pointing: An Unsupervised Learning Approach](https://mlsanity.com/paper/1912.06449/) (2019)

## Built on by (most cited first, 1 of 1)

- [GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representations](https://mlsanity.com/paper/2605.22812/) (2026)

## Abstract (arXiv)

Pointing gestures are a common interaction method used in Human-Robot Collaboration for various tasks, ranging from selecting targets to guiding industrial processes. This study introduces a method for localizing pointed targets within a planar workspace. The approach employs pose estimation, and a simple geometric model based on shoulder-wrist extension to extract gesturing data from an RGB-D stream. The study proposes a rigorous methodology and comprehensive analysis for evaluating pointing gestures and target selection in typical robotic tasks. In addition to evaluating tool accuracy, the tool is integrated into a proof-of-concept robotic system, which includes object detection, speech transcription, and speech synthesis to demonstrate the integration of multiple modalities in a collaborative application. Finally, a discussion over tool limitations and performance is provided to understand its role in multimodal robotic systems. All developments are available at: https://github.com/NMKsas/gesture_pointer.git.