# Multi-Scale Structure-Aware Network for Human Pose Estimation

arXiv: https://arxiv.org/abs/1803.09894 · HTML: https://mlsanity.com/paper/1803.09894/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2018.

## Built on by (most cited first, 30 of 37)

- [Sequential Modeling Enables Scalable Learning for Large Vision Models](https://mlsanity.com/paper/2312.00785/) (2023)
- [Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression](https://mlsanity.com/paper/2104.02300/) (2021)
- [HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network](https://mlsanity.com/paper/2203.14564/) (2022)
- [Online Knowledge Distillation for Efficient Pose Estimation](https://mlsanity.com/paper/2108.02092/) (2021)
- [Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective](https://mlsanity.com/paper/2104.11536/) (2021)
- [TFPose: Direct Human Pose Estimation with Transformers](https://mlsanity.com/paper/2103.15320/) (2021)
- [Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking](https://mlsanity.com/paper/2101.08466/) (2021)
- [PPT: token-Pruned Pose Transformer for monocular and multi-view human pose estimation](https://mlsanity.com/paper/2209.08194/) (2022)
- [Omni-supervised Point Cloud Segmentation via Gradual Receptive Field Component Reasoning](https://mlsanity.com/paper/2105.10203/) (2021)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [A Unified Framework for Domain Adaptive Pose Estimation](https://mlsanity.com/paper/2204.00172/) (2022)
- [Mutual Information-Based Temporal Difference Learning for Human Pose Estimation in Video](https://mlsanity.com/paper/2303.08475/) (2023)
- [SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation](https://mlsanity.com/paper/2205.13490/) (2022)
- [A Comprehensive Review on Deep Supervision: Theories and Applications](https://mlsanity.com/paper/2207.02376/) (2022)
- [Semi-Supervised 2D Human Pose Estimation Driven by Position Inconsistency Pseudo Label Correction Module](https://mlsanity.com/paper/2303.04346/) (2023)
- [Learning to Estimate 3D Human Pose from Point Cloud](https://mlsanity.com/paper/2212.12910/) (2022)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [Single Person Pose Estimation: A Survey](https://mlsanity.com/paper/2109.10056/) (2021)
- [Estimating Pose from Pressure Data for Smart Beds with Deep Image-based Pose Estimators](https://mlsanity.com/paper/2206.06518/) (2022)
- [HandGCAT: Occlusion-Robust 3D Hand Mesh Reconstruction from Monocular Images](https://mlsanity.com/paper/2403.07912/) (2024)
- [Pose-MUM : Reinforcing Key Points Relationship for Semi-Supervised Human Pose Estimation](https://mlsanity.com/paper/2203.07837/) (2022)
- [2D Human Pose Estimation with Explicit Anatomical Keypoints Structure Constraints](https://mlsanity.com/paper/2212.02163/) (2022)
- [Occluded Human Pose Estimation based on Limb Joint Augmentation](https://mlsanity.com/paper/2410.09885/) (2024)
- [Learning Structure-Supporting Dependencies via Keypoint Interactive Transformer for General Mammal Pose Estimation](https://mlsanity.com/paper/2502.18214/) (2025)
- [Data Augmentation in Human-Centric Vision](https://mlsanity.com/paper/2403.08650/) (2024)
- [FasterPose: A Faster Simple Baseline for Human Pose Estimation](https://mlsanity.com/paper/2107.03215/) (2021)
- [Location-free Human Pose Estimation](https://mlsanity.com/paper/2205.12619/) (2022)
- [OVeNet: Offset Vector Network for Semantic Segmentation](https://mlsanity.com/paper/2303.14516/) (2023)
- [Composite Localization for Human Pose Estimation](https://mlsanity.com/paper/2105.07245/) (2021)
- [Self-Supervision and Spatial-Sequential Attention Based Loss for Multi-Person Pose Estimation](https://mlsanity.com/paper/2110.10734/) (2021)

## Abstract (arXiv)

We develop a robust multi-scale structure-aware neural network for human pose estimation. This method improves the recent deep conv-deconv hourglass models with four key improvements: (1) multi-scale supervision to strengthen contextual feature learning in matching body keypoints by combining feature heatmaps across scales, (2) multi-scale regression network at the end to globally optimize the structural matching of the multi-scale features, (3) structure-aware loss used in the intermediate supervision and at the regression to improve the matching of keypoints and respective neighbors to infer a higher-order matching configurations, and (4) a keypoint masking training scheme that can effectively fine-tune our network to robustly localize occluded keypoints via adjacent matches. Our method can effectively improve state-of-the-art pose estimation methods that suffer from difficulties in scale varieties, occlusions, and complex multi-person scenarios. This multi-scale supervision tightly integrates with the regression network to effectively (i) localize keypoints using the ensemble of multi-scale features, and (ii) infer global pose configuration by maximizing structural consistencies across multiple keypoints and scales. The keypoint masking training enhances these advantages to focus learning on hard occlusion samples. Our method achieves the leading position in the MPII challenge leaderboard among the state-of-the-art methods.