# Robustness Evaluation in Hand Pose Estimation Models using Metamorphic Testing

arXiv: https://arxiv.org/abs/2303.04566 · HTML: https://mlsanity.com/paper/2303.04566/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 8 Mar 2023.

## Builds on (most cited first, 10 of 10)

- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [Benchmarking and Error Diagnosis in Multi-Instance Pose Estimation](https://mlsanity.com/paper/1707.05388/) (2017)
- [Fairness Evaluation in Deepfake Detection Models using Metamorphic Testing](https://mlsanity.com/paper/2203.06825/) (2022)
- [Nonparametric Structure Regularization Machine for 2D Hand Pose Estimation](https://mlsanity.com/paper/2001.08869/) (2020)
- [METTLE: a METamorphic testing approach to assessing and validating unsupervised machine LEarning systems](https://mlsanity.com/paper/1807.10453/) (2018)
- [Robustness Evaluation of Stacked Generative Adversarial Networks using Metamorphic Testing](https://mlsanity.com/paper/2103.02870/) (2021)
- [Metamorphic Testing-based Adversarial Attack to Fool Deepfake Detectors](https://mlsanity.com/paper/2204.08612/) (2022)

## Built on by (most cited first, 2 of 2)

- [Siformer: Feature-isolated Transformer for Efficient Skeleton-based Sign Language Recognition](https://mlsanity.com/paper/2503.20436/) (2025)
- [Metamorphic Testing for Pose Estimation Systems](https://mlsanity.com/paper/2502.09460/) (2025)

## Abstract (arXiv)

Hand pose estimation (HPE) is a task that predicts and describes the hand poses from images or video frames. When HPE models estimate hand poses captured in a laboratory or under controlled environments, they normally deliver good performance. However, the real-world environment is complex, and various uncertainties may happen, which could degrade the performance of HPE models. For example, the hands could be occluded, the visibility of hands could be reduced by imperfect exposure rate, and the contour of hands prone to be blurred during fast hand movements. In this work, we adopt metamorphic testing to evaluate the robustness of HPE models and provide suggestions on the choice of HPE models for different applications. The robustness evaluation was conducted on four state-of-the-art models, namely MediaPipe hands, OpenPose, BodyHands, and NSRM hand. We found that on average more than 80% of the hands could not be identified by BodyHands, and at least 50% of hands could not be identified by MediaPipe hands when diagonal motion blur is introduced, while an average of more than 50% of strongly underexposed hands could not be correctly estimated by NSRM hand. Similarly, applying occlusions on only four hand joints will also largely degrade the performance of these models. The experimental results show that occlusions, illumination variations, and motion blur are the main obstacles to the performance of existing HPE models. These findings may pave the way for researchers to improve the performance and robustness of hand pose estimation models and their applications.