# Metamorphic Testing for Pose Estimation Systems

arXiv: https://arxiv.org/abs/2502.09460 · HTML: https://mlsanity.com/paper/2502.09460/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Builds on (most cited first, 22 of 22)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Visual Spatial Reasoning](https://mlsanity.com/paper/2205.00363/) (2022)
- [DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems](https://mlsanity.com/paper/1803.07519/) (2018)
- [BlazePose: On-device Real-time Body Pose tracking](https://mlsanity.com/paper/2006.10204/) (2020)
- [2D/3D Pose Estimation and Action Recognition using Multitask Deep Learning](https://mlsanity.com/paper/1802.09232/) (2018)
- [Taxonomy of Real Faults in Deep Learning Systems](https://mlsanity.com/paper/1910.11015/) (2019)
- [Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing](https://mlsanity.com/paper/2001.00964/) (2020)
- [DeepMutation: Mutation Testing of Deep Learning Systems](https://mlsanity.com/paper/1805.05206/) (2018)
- [Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testing](https://mlsanity.com/paper/1808.05353/) (2018)
- [The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models](https://mlsanity.com/paper/2405.04623/) (2024)
- [BlazePose GHUM Holistic: Real-time 3D Human Landmarks and Pose Estimation](https://mlsanity.com/paper/2206.11678/) (2022)
- [RobustCLEVR: A Benchmark and Framework for Evaluating Robustness in Object-centric Learning](https://mlsanity.com/paper/2308.14899/) (2023)
- [What's left can't be right -- The remaining positional incompetence of contrastive vision-language models](https://mlsanity.com/paper/2311.11477/) (2023)
- [Evaluating Human Trajectory Prediction with Metamorphic Testing](https://mlsanity.com/paper/2407.18756/) (2024)
- [Automated Tackle Injury Risk Assessment in Contact-Based Sports -- A Rugby Union Example](https://mlsanity.com/paper/2104.10916/) (2021)
- [Robustness Evaluation in Hand Pose Estimation Models using Metamorphic Testing](https://mlsanity.com/paper/2303.04566/) (2023)
- [Combating Missed Recalls in E-commerce Search: A CoT-Prompting Testing Approach](https://mlsanity.com/paper/2406.19633/) (2024)

## Built on by (most cited first, 3 of 3)

- [Understanding Space Is Rocket Science -- Only Top Reasoning Models Can Solve Spatial Understanding Tasks](https://mlsanity.com/paper/2509.02175/) (2025)
- [Identifying Ethical Biases in Action Recognition Models](https://mlsanity.com/paper/2604.17971/) (2026)
- [Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning](https://mlsanity.com/paper/2606.06056/) (2026)

## Abstract (arXiv)

Pose estimation systems are used in a variety of fields, from sports analytics to livestock care. Given their potential impact, it is paramount to systematically test their behaviour and potential for failure. This is a complex task due to the oracle problem and the high cost of manual labelling necessary to build ground truth keypoints. This problem is exacerbated by the fact that different applications require systems to focus on different subjects (e.g., human versus animal) or landmarks (e.g., only extremities versus whole body and face), which makes labelled test data rarely reusable. To combat these problems we propose MET-POSE, a metamorphic testing framework for pose estimation systems that bypasses the need for manual annotation while assessing the performance of these systems under different circumstances. MET-POSE thus allows users of pose estimation systems to assess the systems in conditions that more closely relate to their application without having to label an ad-hoc test dataset or rely only on available datasets, which may not be adapted to their application domain. While we define MET-POSE in general terms, we also present a non-exhaustive list of metamorphic rules that represent common challenges in computer vision applications, as well as a specific way to evaluate these rules. We then experimentally show the effectiveness of MET-POSE by applying it to Mediapipe Holistic, a state of the art human pose estimation system, with the FLIC and PHOENIX datasets. With these experiments, we outline numerous ways in which the outputs of MET-POSE can uncover faults in pose estimation systems at a similar or higher rate than classic testing using hand labelled data, and show that users can tailor the rule set they use to the faults and level of accuracy relevant to their application.