# Real-time Human Finger Pointing Recognition and Estimation for Robot Directives Using a Single Web-Camera

arXiv: https://arxiv.org/abs/2307.02949 · HTML: https://mlsanity.com/paper/2307.02949/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2023.

## Built on by (most cited first, 4 of 4)

- [Ultra-Range Gesture Recognition using a Web-Camera in Human-Robot Interaction](https://mlsanity.com/paper/2311.15361/) (2023)
- [Pointing-Guided Target Estimation via Transformer-Based Attention](https://mlsanity.com/paper/2509.05031/) (2025)
- [Recognition of Dynamic Hand Gestures in Long Distance using a Web-Camera for Robot Guidance](https://mlsanity.com/paper/2406.12424/) (2024)
- [DiG-Net: Enhancing Human-Robot Interaction through Hyper-Range Dynamic Gesture Recognition in Assistive Robotics](https://mlsanity.com/paper/2505.24786/) (2025)

## Abstract (arXiv)

Gestures play a pivotal role in human communication, often serving as a preferred or complementary medium to verbal expression due to their superior spatial reference capabilities. A finger-pointing gesture conveys vital information regarding some point of interest in the environment. In Human-Robot Interaction (HRI), users can easily direct robots to target locations, facilitating tasks in diverse domains such as search and rescue or factory assistance. State-of-the-art approaches for visual pointing estimation often rely on depth cameras, are limited to indoor environments, and provide discrete predictions between limited targets. In this paper, we explore the development of models that enable robots to understand pointing directives from humans using a single web camera, even in diverse indoor and outdoor environments. A novel perception framework is proposed which includes a designated data-based model termed PointingNet. PointingNet recognizes the occurrence of pointing through classification followed by approximating the position and direction of the index finger with an advanced regression model. The model relies on a novel segmentation model for masking any lifted arm. While state-of-the-art human pose estimation models provide poor pointing angle estimation error of 28deg, PointingNet exhibits a mean error of less than 2deg. With the pointing information, the target location is computed, followed by robot motion planning and execution. The framework is evaluated on two robotic systems, demonstrating accurate target reaching.