# Self-Correctable and Adaptable Inference for Generalizable Human Pose Estimation

arXiv: https://arxiv.org/abs/2303.11180 · HTML: https://mlsanity.com/paper/2303.11180/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 47 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 30 of 41)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [Distribution-Aware Coordinate Representation for Human Pose Estimation](https://mlsanity.com/paper/1910.06278/) (2019)
- [Dual Learning for Machine Translation](https://mlsanity.com/paper/1611.00179/) (2016)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [Human Pose Estimation with Iterative Error Feedback](https://mlsanity.com/paper/1507.06550/) (2015)
- [Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression](https://mlsanity.com/paper/2104.02300/) (2021)
- [Self-supervised Learning of Motion Capture](https://mlsanity.com/paper/1712.01337/) (2017)
- [Learning Actor Relation Graphs for Group Activity Recognition](https://mlsanity.com/paper/1904.10117/) (2019)
- [The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation](https://mlsanity.com/paper/1911.07524/) (2019)
- [Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation](https://mlsanity.com/paper/2012.15175/) (2020)
- [P-CNN: Pose-based CNN Features for Action Recognition](https://mlsanity.com/paper/1506.03607/) (2015)
- [Rethinking on Multi-Stage Networks for Human Pose Estimation](https://mlsanity.com/paper/1901.00148/) (2019)
- [Combining detection and tracking for human pose estimation in videos](https://mlsanity.com/paper/2003.13743/) (2020)
- [PoseFix: Model-agnostic General Human Pose Refinement Network](https://mlsanity.com/paper/1812.03595/) (2018)
- [Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition](https://mlsanity.com/paper/1611.09078/) (2016)
- [Learning Dynamics via Graph Neural Networks for Human Pose Estimation and Tracking](https://mlsanity.com/paper/2106.03772/) (2021)
- [Multi-Person Pose Estimation with Enhanced Channel-wise and Spatial Information](https://mlsanity.com/paper/1905.03466/) (2019)
- [Dual Supervised Learning](https://mlsanity.com/paper/1707.00415/) (2017)

## Built on by (most cited first, 7 of 7)

- [Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models](https://mlsanity.com/paper/2410.12790/) (2024)
- [Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioning](https://mlsanity.com/paper/2404.05218/) (2024)
- [Cross-Modal Concept Learning and Inference for Vision-Language Models](https://mlsanity.com/paper/2307.15460/) (2023)
- [Learning Visual Conditioning Tokens to Correct Domain Shift for Fully Test-time Adaptation](https://mlsanity.com/paper/2406.19341/) (2024)
- [Dual-Path Adversarial Lifting for Domain Shift Correction in Online Test-time Adaptation](https://mlsanity.com/paper/2408.13983/) (2024)
- [RSB-Pose: Robust Short-Baseline Binocular 3D Human Pose Estimation with Occlusion Handling](https://mlsanity.com/paper/2311.14242/) (2023)
- [Cross-Domain Knowledge Distillation for Low-Resolution Human Pose Estimation](https://mlsanity.com/paper/2405.11448/) (2024)

## Abstract (arXiv)

A central challenge in human pose estimation, as well as in many other machine learning and prediction tasks, is the generalization problem. The learned network does not have the capability to characterize the prediction error, generate feedback information from the test sample, and correct the prediction error on the fly for each individual test sample, which results in degraded performance in generalization. In this work, we introduce a self-correctable and adaptable inference (SCAI) method to address the generalization challenge of network prediction and use human pose estimation as an example to demonstrate its effectiveness and performance. We learn a correction network to correct the prediction result conditioned by a fitness feedback error. This feedback error is generated by a learned fitness feedback network which maps the prediction result to the original input domain and compares it against the original input. Interestingly, we find that this self-referential feedback error is highly correlated with the actual prediction error. This strong correlation suggests that we can use this error as feedback to guide the correction process. It can be also used as a loss function to quickly adapt and optimize the correction network during the inference process. Our extensive experimental results on human pose estimation demonstrate that the proposed SCAI method is able to significantly improve the generalization capability and performance of human pose estimation.