# MDPose: Real-Time Multi-Person Pose Estimation via Mixture Density Model

arXiv: https://arxiv.org/abs/2302.08751 · HTML: https://mlsanity.com/paper/2302.08751/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 36 references that are in the map.
- First posted on arXiv 17 Feb 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [Human Pose Regression with Residual Log-likelihood Estimation](https://mlsanity.com/paper/2107.11291/) (2021)
- [PifPaf: Composite Fields for Human Pose Estimation](https://mlsanity.com/paper/1903.06593/) (2019)
- [Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression](https://mlsanity.com/paper/2104.02300/) (2021)
- [Pose2Seg: Detection Free Human Instance Segmentation](https://mlsanity.com/paper/1803.10683/) (2018)
- [Generating Multiple Hypotheses for 3D Human Pose Estimation with Mixture Density Network](https://mlsanity.com/paper/1904.05547/) (2019)
- [MegDet: A Large Mini-Batch Object Detector](https://mlsanity.com/paper/1711.07240/) (2017)
- [Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation](https://mlsanity.com/paper/2007.11864/) (2020)
- [DirectPose: Direct End-to-End Multi-Person Pose Estimation](https://mlsanity.com/paper/1911.07451/) (2019)
- [Deep Directional Statistics: Pose Estimation with Uncertainty Quantification](https://mlsanity.com/paper/1805.03430/) (2018)
- [FCPose: Fully Convolutional Multi-Person Pose Estimation with Dynamic Instance-Aware Convolutions](https://mlsanity.com/paper/2105.14185/) (2021)
- [Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation](https://mlsanity.com/paper/2101.11223/) (2021)
- [Mixture Dense Regression for Object Detection and Human Pose Estimation](https://mlsanity.com/paper/1912.00821/) (2019)
- [Peeking into occluded joints: A novel framework for crowd pose estimation](https://mlsanity.com/paper/2003.10506/) (2020)
- [Learning Local-Global Contextual Adaptation for Multi-Person Pose Estimation](https://mlsanity.com/paper/2109.03622/) (2021)

## Built on by (most cited first, 1 of 1)

- [DIR-BHRNet: A Lightweight Network for Real-time Vision-based Multi-person Pose Estimation on Smartphones](https://mlsanity.com/paper/2407.13777/) (2024)

## Abstract (arXiv)

One of the major challenges in multi-person pose estimation is instance-aware keypoint estimation. Previous methods address this problem by leveraging an off-the-shelf detector, heuristic post-grouping process or explicit instance identification process, hindering further improvements in the inference speed which is an important factor for practical applications. From the statistical point of view, those additional processes for identifying instances are necessary to bypass learning the high-dimensional joint distribution of human keypoints, which is a critical factor for another major challenge, the occlusion scenario. In this work, we propose a novel framework of single-stage instance-aware pose estimation by modeling the joint distribution of human keypoints with a mixture density model, termed as MDPose. Our MDPose estimates the distribution of human keypoints' coordinates using a mixture density model with an instance-aware keypoint head consisting simply of 8 convolutional layers. It is trained by minimizing the negative log-likelihood of the ground truth keypoints. Also, we propose a simple yet effective training strategy, Random Keypoint Grouping (RKG), which significantly alleviates the underflow problem leading to successful learning of relations between keypoints. On OCHuman dataset, which consists of images with highly occluded people, our MDPose achieves state-of-the-art performance by successfully learning the high-dimensional joint distribution of human keypoints. Furthermore, our MDPose shows significant improvement in inference speed with a competitive accuracy on MS COCO, a widely-used human keypoint dataset, thanks to the proposed much simpler single-stage pipeline.