# Hand tracking for clinical applications: validation of the Google MediaPipe Hand (GMH) and the depth-enhanced GMH-D frameworks

arXiv: https://arxiv.org/abs/2308.01088 · HTML: https://mlsanity.com/paper/2308.01088/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 119 references that are in the map.
- First posted on arXiv 2 Aug 2023.

## Builds on (most cited first, 16 of 16)

- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Learning to Estimate 3D Hand Pose from Single RGB Images](https://mlsanity.com/paper/1705.01389/) (2017)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB](https://mlsanity.com/paper/1712.01057/) (2017)
- [Cross-modal Deep Variational Hand Pose Estimation](https://mlsanity.com/paper/1803.11404/) (2018)
- [Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor](https://mlsanity.com/paper/1704.02201/) (2017)
- [Fast and Robust Hand Tracking Using Detection-Guided Optimization](https://mlsanity.com/paper/1602.04124/) (2016)
- [Model-based Deep Hand Pose Estimation](https://mlsanity.com/paper/1606.06854/) (2016)
- [Attention! A Lightweight 2D Hand Pose Estimation Approach](https://mlsanity.com/paper/2001.08047/) (2020)
- [Hand tracking for immersive virtual reality: opportunities and challenges](https://mlsanity.com/paper/2103.14853/) (2021)
- [Towards a Comprehensive Solution for a Vision-based Digitized Neurological Examination](https://mlsanity.com/paper/2205.07209/) (2022)
- [Accurate Hand Keypoint Localization on Mobile Devices](https://mlsanity.com/paper/1812.08028/) (2018)
- [An End-to-end Framework for Unconstrained Monocular 3D Hand Pose Estimation](https://mlsanity.com/paper/1911.12501/) (2019)
- [3D Hand Pose Estimation via Regularized Graph Representation Learning](https://mlsanity.com/paper/1912.01875/) (2019)

## Built on by (most cited first, 4 of 4)

- [EHWGesture -- A dataset for multimodal understanding of clinical gestures](https://mlsanity.com/paper/2509.07525/) (2025)
- [Interpretable and Granular Video-Based Quantification of Motor Characteristics from the Finger Tapping Test in Parkinson Disease](https://mlsanity.com/paper/2506.18925/) (2025)
- [Biomechanics-aware Multi-view Markerless Motion Capture of Dexterous Hand Movements](https://mlsanity.com/paper/2607.02796/) (2026)
- [HandSplatter: Automated Digital Goniometry from Neural Rendering](https://mlsanity.com/paper/2608.09735/) (2026)

## Abstract (arXiv)

Accurate 3D tracking of hand and fingers movements poses significant challenges in computer vision. The potential applications span across multiple domains, including human-computer interaction, virtual reality, industry, and medicine. While gesture recognition has achieved remarkable accuracy, quantifying fine movements remains a hurdle, particularly in clinical applications where the assessment of hand dysfunctions and rehabilitation training outcomes necessitate precise measurements. Several novel and lightweight frameworks based on Deep Learning have emerged to address this issue; however, their performance in accurately and reliably measuring fingers movements requires validation against well-established gold standard systems. In this paper, the aim is to validate the handtracking framework implemented by Google MediaPipe Hand (GMH) and an innovative enhanced version, GMH-D, that exploits the depth estimation of an RGB-Depth camera to achieve more accurate tracking of 3D movements. Three dynamic exercises commonly administered by clinicians to assess hand dysfunctions, namely Hand Opening-Closing, Single Finger Tapping and Multiple Finger Tapping are considered. Results demonstrate high temporal and spectral consistency of both frameworks with the gold standard. However, the enhanced GMH-D framework exhibits superior accuracy in spatial measurements compared to the baseline GMH, for both slow and fast movements. Overall, our study contributes to the advancement of hand tracking technology, the establishment of a validation procedure as a good-practice to prove efficacy of deep-learning-based hand-tracking, and proves the effectiveness of GMH-D as a reliable framework for assessing 3D hand movements in clinical applications.