# Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation

arXiv: https://arxiv.org/abs/2111.08557 · HTML: https://mlsanity.com/paper/2111.08557/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 72 of its 83 references that are in the map.
- First posted on arXiv 16 Nov 2021.

## Builds on (most cited first, 30 of 72)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [CSPNet: A New Backbone that can Enhance Learning Capability of CNN](https://mlsanity.com/paper/1911.11929/) (2019)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)

## Built on by (most cited first, 20 of 20)

- [RTMO: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation](https://mlsanity.com/paper/2312.07526/) (2023)
- [FenceNet: Fine-grained Footwork Recognition in Fencing](https://mlsanity.com/paper/2204.09434/) (2022)
- [YOLOv1 to YOLOv10: The fastest and most accurate real-time object detection systems](https://mlsanity.com/paper/2408.09332/) (2024)
- [Referring Human Pose and Mask Estimation in the Wild](https://mlsanity.com/paper/2410.20508/) (2024)
- [Underground Mapping and Localization Based on Ground-Penetrating Radar](https://mlsanity.com/paper/2409.16446/) (2024)
- [3D Plant Root Skeleton Detection and Extraction](https://mlsanity.com/paper/2508.08094/) (2025)
- [Occluded Human Pose Estimation based on Limb Joint Augmentation](https://mlsanity.com/paper/2410.09885/) (2024)
- [A View Independent Classification Framework for Yoga Postures](https://mlsanity.com/paper/2206.13577/) (2022)
- [Single-stage Multi-human Parsing via Point Sets and Center-based Offsets](https://mlsanity.com/paper/2304.11356/) (2023)
- [BoIR: Box-Supervised Instance Representation for Multi-Person Pose Estimation](https://mlsanity.com/paper/2309.14072/) (2023)
- [SDPose: Tokenized Pose Estimation via Circulation-Guide Self-Distillation](https://mlsanity.com/paper/2404.03518/) (2024)
- [GatedUniPose: A Novel Approach for Pose Estimation Combining UniRepLKNet and Gated Convolution](https://mlsanity.com/paper/2409.07752/) (2024)
- [MamKPD: A Simple Mamba Baseline for Real-Time 2D Keypoint Detection](https://mlsanity.com/paper/2412.01422/) (2024)
- [DETRPose: Real-Time End-to-End Multi-Person Pose Estimation via Modified Transformer Decoder and Novel Denoising Keypoints](https://mlsanity.com/paper/2506.13027/) (2025)
- [Intra-DP: A High Performance Collaborative Inference System for Mobile Edge Computing](https://mlsanity.com/paper/2507.05829/) (2025)
- [Entropy-driven Unsupervised Keypoint Representation Learning in Videos](https://mlsanity.com/paper/2209.15404/) (2022)
- [Hybrid model for Single-Stage Multi-Person Pose Estimation](https://mlsanity.com/paper/2305.01167/) (2023)
- [Non-Contact Breathing Rate Detection Using Optical Flow](https://mlsanity.com/paper/2311.08426/) (2023)
- [FlyPose: Towards Robust Human Pose Estimation From Aerial Views](https://mlsanity.com/paper/2601.05747/) (2026)
- [ER-Pose: Rethinking Keypoint-Driven Representation Learning for Real-Time Human Pose Estimation](https://mlsanity.com/paper/2603.08681/) (2026)

## Abstract (arXiv)

In keypoint estimation tasks such as human pose estimation, heatmap-based regression is the dominant approach despite possessing notable drawbacks: heatmaps intrinsically suffer from quantization error and require excessive computation to generate and post-process. Motivated to find a more efficient solution, we propose to model individual keypoints and sets of spatially related keypoints (i.e., poses) as objects within a dense single-stage anchor-based detection framework. Hence, we call our method KAPAO (pronounced "Ka-Pow"), for Keypoints And Poses As Objects. KAPAO is applied to the problem of single-stage multi-person human pose estimation by simultaneously detecting human pose and keypoint objects and fusing the detections to exploit the strengths of both object representations. In experiments, we observe that KAPAO is faster and more accurate than previous methods, which suffer greatly from heatmap post-processing. The accuracy-speed trade-off is especially favourable in the practical setting when not using test-time augmentation. Source code: https://github.com/wmcnally/kapao.