# On-device Real-time Hand Gesture Recognition

arXiv: https://arxiv.org/abs/2111.00038 · HTML: https://mlsanity.com/paper/2111.00038/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 13 references that are in the map.
- First posted on arXiv 29 Oct 2021.

## Builds on (most cited first, 5 of 5)

- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [BlazePose: On-device Real-time Body Pose tracking](https://mlsanity.com/paper/2006.10204/) (2020)
- [Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows](https://mlsanity.com/paper/2003.10350/) (2020)

## Built on by (most cited first, 6 of 6)

- [BlazePose GHUM Holistic: Real-time 3D Human Landmarks and Pose Estimation](https://mlsanity.com/paper/2206.11678/) (2022)
- [On-device Real-time Custom Hand Gesture Recognition](https://mlsanity.com/paper/2309.10858/) (2023)
- [Proving the Potential of Skeleton Based Action Recognition to Automate the Analysis of Manual Processes](https://mlsanity.com/paper/2310.08451/) (2023)
- [GRLib: An Open-Source Hand Gesture Detection and Recognition Python Library](https://mlsanity.com/paper/2310.14919/) (2023)
- [Automated Patient Positioning with Learned 3D Hand Gestures](https://mlsanity.com/paper/2407.14903/) (2024)
- [Beyond Landmark Extraction: A Framework for Robust Geometric Feature Construction in Structured Image Classification](https://mlsanity.com/paper/2609.00634/) (2026)

## Abstract (arXiv)

We present an on-device real-time hand gesture recognition (HGR) system, which detects a set of predefined static gestures from a single RGB camera. The system consists of two parts: a hand skeleton tracker and a gesture classifier. We use MediaPipe Hands as the basis of the hand skeleton tracker, improve the keypoint accuracy, and add the estimation of 3D keypoints in a world metric space. We create two different gesture classifiers, one based on heuristics and the other using neural networks (NN).