# Towards Understanding the Adversarial Vulnerability of Skeleton-based Action Recognition

arXiv: https://arxiv.org/abs/2005.07151 · HTML: https://mlsanity.com/paper/2005.07151/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2020.

## Built on by (most cited first, 12 of 12)

- [MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction](https://mlsanity.com/paper/2108.07152/) (2021)
- [BASAR:Black-box Attack on Skeletal Action Recognition](https://mlsanity.com/paper/2103.05266/) (2021)
- [Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack](https://mlsanity.com/paper/2103.05347/) (2021)
- [Adversarial Bone Length Attack on Action Recognition](https://mlsanity.com/paper/2109.05830/) (2021)
- [Defending Black-box Skeleton-based Human Activity Classifiers](https://mlsanity.com/paper/2203.04713/) (2022)
- [Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack](https://mlsanity.com/paper/2211.11312/) (2022)
- [Three-Stream 3D/1D CNN for Fine-Grained Action Classification and Segmentation in Table Tennis](https://mlsanity.com/paper/2109.14306/) (2021)
- [Anonymization for Skeleton Action Recognition](https://mlsanity.com/paper/2111.15129/) (2021)
- [Post-train Black-box Defense via Bayesian Boundary Correction](https://mlsanity.com/paper/2306.16979/) (2023)
- [MDMP: Multi-modal Diffusion for supervised Motion Predictions with uncertainty](https://mlsanity.com/paper/2410.03860/) (2024)
- [Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions](https://mlsanity.com/paper/2101.06704/) (2021)
- [Emotion Loss Attacking: Adversarial Attack Perception for Skeleton based on Multi-dimensional Features](https://mlsanity.com/paper/2406.19815/) (2024)

## Abstract (arXiv)

Skeleton-based action recognition has attracted increasing attention due to its strong adaptability to dynamic circumstances and potential for broad applications such as autonomous and anonymous surveillance. With the help of deep learning techniques, it has also witnessed substantial progress and currently achieved around 90% accuracy in benign environment. On the other hand, research on the vulnerability of skeleton-based action recognition under different adversarial settings remains scant, which may raise security concerns about deploying such techniques into real-world systems. However, filling this research gap is challenging due to the unique physical constraints of skeletons and human actions. In this paper, we attempt to conduct a thorough study towards understanding the adversarial vulnerability of skeleton-based action recognition. We first formulate generation of adversarial skeleton actions as a constrained optimization problem by representing or approximating the physiological and physical constraints with mathematical formulations. Since the primal optimization problem with equality constraints is intractable, we propose to solve it by optimizing its unconstrained dual problem using ADMM. We then specify an efficient plug-in defense, inspired by recent theories and empirical observations, against the adversarial skeleton actions. Extensive evaluations demonstrate the effectiveness of the attack and defense method under different settings.