# From Movements to Metrics: Evaluating Explainable AI Methods in Skeleton-Based Human Activity Recognition

arXiv: https://arxiv.org/abs/2402.12790 · HTML: https://mlsanity.com/paper/2402.12790/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 16 of 16)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [Grad-CAM: Why did you say that?](https://mlsanity.com/paper/1611.07450/) (2016)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies](https://mlsanity.com/paper/2007.15911/) (2020)
- [Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition](https://mlsanity.com/paper/2010.09978/) (2020)
- [Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition](https://mlsanity.com/paper/2106.15125/) (2021)
- [Rethinking Stability for Attribution-based Explanations](https://mlsanity.com/paper/2203.06877/) (2022)
- [Adversarial Attack on Skeleton-based Human Action Recognition](https://mlsanity.com/paper/1909.06500/) (2019)
- [Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack](https://mlsanity.com/paper/2103.05347/) (2021)

## Built on by (most cited first, 6 of 6)

- [SMART-Vision: Survey of Modern Action Recognition Techniques in Vision](https://mlsanity.com/paper/2501.13066/) (2025)
- [Explaining Human Activity Recognition with SHAP: Validating Insights with Perturbation and Quantitative Measures](https://mlsanity.com/paper/2411.03714/) (2024)
- [Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral Palsy](https://mlsanity.com/paper/2409.00001/) (2024)
- [Choose Your Explanation: A Comparison of SHAP and GradCAM in Human Activity Recognition](https://mlsanity.com/paper/2412.16003/) (2024)
- [Gradient Attention Map Based Verification of Deep Convolutional Neural Networks with Application to X-ray Image Datasets](https://mlsanity.com/paper/2504.21227/) (2025)
- [Explainable Human Activity Recognition: A Unified Review of Concepts and Mechanisms](https://mlsanity.com/paper/2604.09799/) (2026)

## Abstract (arXiv)

The advancement of deep learning in human activity recognition (HAR) using 3D skeleton data is critical for applications in healthcare, security, sports, and human-computer interaction. This paper tackles a well-known gap in the field, which is the lack of testing in the applicability and reliability of XAI evaluation metrics in the skeleton-based HAR domain. We have tested established XAI metrics namely faithfulness and stability on Class Activation Mapping (CAM) and Gradient-weighted Class Activation Mapping (Grad-CAM) to address this problem. The study also introduces a perturbation method that respects human biomechanical constraints to ensure realistic variations in human movement. Our findings indicate that textitfaithfulness may not be a reliable metric in certain contexts, such as with the EfficientGCN model. Conversely, stability emerges as a more dependable metric when there is slight input data perturbations. CAM and Grad-CAM are also found to produce almost identical explanations, leading to very similar XAI metric performance. This calls for the need for more diversified metrics and new XAI methods applied in skeleton-based HAR.