# Poseur: Direct Human Pose Regression with Transformers

arXiv: https://arxiv.org/abs/2201.07412 · HTML: https://mlsanity.com/paper/2201.07412/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 41 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Whole-Body Human Pose Estimation in the Wild](https://mlsanity.com/paper/2007.11858/) (2020)
- [Distribution-Aware Coordinate Representation for Human Pose Estimation](https://mlsanity.com/paper/1910.06278/) (2019)
- [HRFormer: High-Resolution Transformer for Dense Prediction](https://mlsanity.com/paper/2110.09408/) (2021)
- [Human Pose Regression with Residual Log-likelihood Estimation](https://mlsanity.com/paper/2107.11291/) (2021)
- [TransPose: Keypoint Localization via Transformer](https://mlsanity.com/paper/2012.14214/) (2020)
- [Compositional Human Pose Regression](https://mlsanity.com/paper/1704.00159/) (2017)
- [Human Pose Estimation with Iterative Error Feedback](https://mlsanity.com/paper/1507.06550/) (2015)
- [TokenPose: Learning Keypoint Tokens for Human Pose Estimation](https://mlsanity.com/paper/2104.03516/) (2021)
- [Pose Recognition with Cascade Transformers](https://mlsanity.com/paper/2104.06976/) (2021)
- [Single-Stage Multi-Person Pose Machines](https://mlsanity.com/paper/1908.09220/) (2019)
- [The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation](https://mlsanity.com/paper/1911.07524/) (2019)
- [Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation](https://mlsanity.com/paper/2012.15175/) (2020)

## Built on by (most cited first, 30 of 43)

- [RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose](https://mlsanity.com/paper/2303.07399/) (2023)
- [ViTPose++: Vision Transformer for Generic Body Pose Estimation](https://mlsanity.com/paper/2212.04246/) (2022)
- [UniHCP: A Unified Model for Human-Centric Perceptions](https://mlsanity.com/paper/2303.02936/) (2023)
- [Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation](https://mlsanity.com/paper/2302.01593/) (2023)
- [Human Pose as Compositional Tokens](https://mlsanity.com/paper/2303.11638/) (2023)
- [RTMO: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation](https://mlsanity.com/paper/2312.07526/) (2023)
- [Group Pose: A Simple Baseline for End-to-End Multi-person Pose Estimation](https://mlsanity.com/paper/2308.07313/) (2023)
- [FinePOSE: Fine-Grained Prompt-Driven 3D Human Pose Estimation via Diffusion Models](https://mlsanity.com/paper/2405.05216/) (2024)
- [X-Pose: Detecting Any Keypoints](https://mlsanity.com/paper/2310.08530/) (2023)
- [RTMW: Real-Time Multi-Person 2D and 3D Whole-body Pose Estimation](https://mlsanity.com/paper/2407.08634/) (2024)
- [DistilPose: Tokenized Pose Regression with Heatmap Distillation](https://mlsanity.com/paper/2303.02455/) (2023)
- [Neural Interactive Keypoint Detection](https://mlsanity.com/paper/2308.10174/) (2023)
- [On the Calibration of Human Pose Estimation](https://mlsanity.com/paper/2311.17105/) (2023)
- [Video-Based Human Pose Regression via Decoupled Space-Time Aggregation](https://mlsanity.com/paper/2403.19926/) (2024)
- [Capturing the motion of every joint: 3D human pose and shape estimation with independent tokens](https://mlsanity.com/paper/2303.00298/) (2023)
- [Mitigating Motion Blur for Robust 3D Baseball Player Pose Modeling for Pitch Analysis](https://mlsanity.com/paper/2309.01010/) (2023)
- [Lightweight Super-Resolution Head for Human Pose Estimation](https://mlsanity.com/paper/2307.16765/) (2023)
- [Meta-Point Learning and Refining for Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2403.13647/) (2024)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [GTPT: Group-based Token Pruning Transformer for Efficient Human Pose Estimation](https://mlsanity.com/paper/2407.10756/) (2024)
- [Referring Human Pose and Mask Estimation in the Wild](https://mlsanity.com/paper/2410.20508/) (2024)
- [Diversifying Human Pose in Synthetic Data for Aerial-view Human Detection](https://mlsanity.com/paper/2405.15939/) (2024)
- [PoseBench: Benchmarking the Robustness of Pose Estimation Models under Corruptions](https://mlsanity.com/paper/2406.14367/) (2024)
- [COD: Learning Conditional Invariant Representation for Domain Adaptation Regression](https://mlsanity.com/paper/2408.06638/) (2024)
- [VTP: Volumetric Transformer for Multi-view Multi-person 3D Pose Estimation](https://mlsanity.com/paper/2205.12602/) (2022)
- [Bias-Compensated Integral Regression for Human Pose Estimation](https://mlsanity.com/paper/2301.10431/) (2023)
- [Joint Coordinate Regression and Association For Multi-Person Pose Estimation, A Pure Neural Network Approach](https://mlsanity.com/paper/2307.01004/) (2023)
- [Geometry-Biased Transformer for Robust Multi-View 3D Human Pose Reconstruction](https://mlsanity.com/paper/2312.17106/) (2023)
- [Di2Pose: Discrete Diffusion Model for Occluded 3D Human Pose Estimation](https://mlsanity.com/paper/2405.17016/) (2024)
- [Recoverable Anonymization for Pose Estimation: A Privacy-Enhancing Approach](https://mlsanity.com/paper/2409.02715/) (2024)

## Abstract (arXiv)

We propose a direct, regression-based approach to 2D human pose estimation from single images. We formulate the problem as a sequence prediction task, which we solve using a Transformer network. This network directly learns a regression mapping from images to the keypoint coordinates, without resorting to intermediate representations such as heatmaps. This approach avoids much of the complexity associated with heatmap-based approaches. To overcome the feature misalignment issues of previous regression-based methods, we propose an attention mechanism that adaptively attends to the features that are most relevant to the target keypoints, considerably improving the accuracy. Importantly, our framework is end-to-end differentiable, and naturally learns to exploit the dependencies between keypoints. Experiments on MS-COCO and MPII, two predominant pose-estimation datasets, demonstrate that our method significantly improves upon the state-of-the-art in regression-based pose estimation. More notably, ours is the first regression-based approach to perform favorably compared to the best heatmap-based pose estimation methods.