# When Human Pose Estimation Meets Robustness: Adversarial Algorithms and Benchmarks

arXiv: https://arxiv.org/abs/2105.06152 · HTML: https://mlsanity.com/paper/2105.06152/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 51 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 30 of 39)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [A Fourier Perspective on Model Robustness in Computer Vision](https://mlsanity.com/paper/1906.08988/) (2019)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [Whole-Body Human Pose Estimation in the Wild](https://mlsanity.com/paper/2007.11858/) (2020)
- [Improving the Robustness of Deep Neural Networks via Stability Training](https://mlsanity.com/paper/1604.04326/) (2016)
- [GridMask Data Augmentation](https://mlsanity.com/paper/2001.04086/) (2020)
- [PifPaf: Composite Fields for Human Pose Estimation](https://mlsanity.com/paper/1903.06593/) (2019)
- [Understanding How Image Quality Affects Deep Neural Networks](https://mlsanity.com/paper/1604.04004/) (2016)
- [Benchmarking the Robustness of Semantic Segmentation Models](https://mlsanity.com/paper/1908.05005/) (2019)
- [Adversarial AutoAugment](https://mlsanity.com/paper/1912.11188/) (2019)
- [Pose2Seg: Detection Free Human Instance Segmentation](https://mlsanity.com/paper/1803.10683/) (2018)

## Built on by (most cited first, 30 of 41)

- [Robo3D: Towards Robust and Reliable 3D Perception against Corruptions](https://mlsanity.com/paper/2303.17597/) (2023)
- [Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformer](https://mlsanity.com/paper/2204.08680/) (2022)
- [AugMax: Adversarial Composition of Random Augmentations for Robust Training](https://mlsanity.com/paper/2110.13771/) (2021)
- [On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2201.05057/) (2022)
- [RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions](https://mlsanity.com/paper/2310.15171/) (2023)
- [Benchmarking the Robustness of Spatial-Temporal Models Against Corruptions](https://mlsanity.com/paper/2110.06513/) (2021)
- [Perceptual Attacks of No-Reference Image Quality Models with Human-in-the-Loop](https://mlsanity.com/paper/2210.00933/) (2022)
- [Benchmarking the Robustness of Deep Neural Networks to Common Corruptions in Digital Pathology](https://mlsanity.com/paper/2206.14973/) (2022)
- [Teacher-Student Architecture for Knowledge Distillation: A Survey](https://mlsanity.com/paper/2308.04268/) (2023)
- [Diffusion Model is a Good Pose Estimator from 3D RF-Vision](https://mlsanity.com/paper/2403.16198/) (2024)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [Vision-Based Human Pose Estimation via Deep Learning: A Survey](https://mlsanity.com/paper/2308.13872/) (2023)
- [Intra-Source Style Augmentation for Improved Domain Generalization](https://mlsanity.com/paper/2210.10175/) (2022)
- [Distracting Downpour: Adversarial Weather Attacks for Motion Estimation](https://mlsanity.com/paper/2305.06716/) (2023)
- [Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors](https://mlsanity.com/paper/2408.03200/) (2024)
- [PoseTrans: A Simple Yet Effective Pose Transformation Augmentation for Human Pose Estimation](https://mlsanity.com/paper/2208.07755/) (2022)
- [The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation](https://mlsanity.com/paper/2307.15061/) (2023)
- [Benchmarks for Corruption Invariant Person Re-identification](https://mlsanity.com/paper/2111.00880/) (2021)
- [Attacking Motion Estimation with Adversarial Snow](https://mlsanity.com/paper/2210.11242/) (2022)
- [Anatomy-guided domain adaptation for 3D in-bed human pose estimation](https://mlsanity.com/paper/2211.12193/) (2022)
- [You Only Learn One Query: Learning Unified Human Query for Single-Stage Multi-Person Multi-Task Human-Centric Perception](https://mlsanity.com/paper/2312.05525/) (2023)
- [Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization](https://mlsanity.com/paper/2307.00648/) (2023)
- [KBody: Towards general, robust, and aligned monocular whole-body estimation](https://mlsanity.com/paper/2304.11542/) (2023)
- [Towards Robust and Expressive Whole-body Human Pose and Shape Estimation](https://mlsanity.com/paper/2312.08730/) (2023)
- [Adversarial Training: A Survey](https://mlsanity.com/paper/2410.15042/) (2024)
- [Non-Autoregressive Sign Language Production via Knowledge Distillation](https://mlsanity.com/paper/2208.06183/) (2022)
- [Improving the Robustness of 3D Human Pose Estimation: A Benchmark and Learning from Noisy Input](https://mlsanity.com/paper/2312.06797/) (2023)
- [Towards Robust 3D Pose Transfer with Adversarial Learning](https://mlsanity.com/paper/2404.02242/) (2024)
- [PoseBench: Benchmarking the Robustness of Pose Estimation Models under Corruptions](https://mlsanity.com/paper/2406.14367/) (2024)
- [Data Augmentation in Human-Centric Vision](https://mlsanity.com/paper/2403.08650/) (2024)

## Abstract (arXiv)

Human pose estimation is a fundamental yet challenging task in computer vision, which aims at localizing human anatomical keypoints. However, unlike human vision that is robust to various data corruptions such as blur and pixelation, current pose estimators are easily confused by these corruptions. This work comprehensively studies and addresses this problem by building rigorous robust benchmarks, termed COCO-C, MPII-C, and OCHuman-C, to evaluate the weaknesses of current advanced pose estimators, and a new algorithm termed AdvMix is proposed to improve their robustness in different corruptions. Our work has several unique benefits. (1) AdvMix is model-agnostic and capable in a wide-spectrum of pose estimation models. (2) AdvMix consists of adversarial augmentation and knowledge distillation. Adversarial augmentation contains two neural network modules that are trained jointly and competitively in an adversarial manner, where a generator network mixes different corrupted images to confuse a pose estimator, improving the robustness of the pose estimator by learning from harder samples. To compensate for the noise patterns by adversarial augmentation, knowledge distillation is applied to transfer clean pose structure knowledge to the target pose estimator. (3) Extensive experiments show that AdvMix significantly increases the robustness of pose estimations across a wide range of corruptions, while maintaining accuracy on clean data in various challenging benchmark datasets.