# 15 Keypoints Is All You Need

arXiv: https://arxiv.org/abs/1912.02323 · HTML: https://mlsanity.com/paper/1912.02323/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2019.

## Built on by (most cited first, 23 of 23)

- [AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time](https://mlsanity.com/paper/2211.03375/) (2022)
- [TokenPose: Learning Keypoint Tokens for Human Pose Estimation](https://mlsanity.com/paper/2104.03516/) (2021)
- [Do Different Tracking Tasks Require Different Appearance Models?](https://mlsanity.com/paper/2107.02156/) (2021)
- [OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association](https://mlsanity.com/paper/2103.02440/) (2021)
- [Deep Dual Consecutive Network for Human Pose Estimation](https://mlsanity.com/paper/2103.07254/) (2021)
- [Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective](https://mlsanity.com/paper/2104.11536/) (2021)
- [Learning Dynamics via Graph Neural Networks for Human Pose Estimation and Tracking](https://mlsanity.com/paper/2106.03772/) (2021)
- [JRDB-Pose: A Large-scale Dataset for Multi-Person Pose Estimation and Tracking](https://mlsanity.com/paper/2210.11940/) (2022)
- [Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2203.15227/) (2022)
- [Tracking People with 3D Representations](https://mlsanity.com/paper/2111.07868/) (2021)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021)
- [Unified Keypoint-based Action Recognition Framework via Structured Keypoint Pooling](https://mlsanity.com/paper/2303.15270/) (2023)
- [Learning to Track Instances without Video Annotations](https://mlsanity.com/paper/2104.00287/) (2021)
- [The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person Pose Estimation](https://mlsanity.com/paper/2110.05132/) (2021)
- [Tracking People by Predicting 3D Appearance, Location & Pose](https://mlsanity.com/paper/2112.04477/) (2021)
- [Learning to Estimate External Forces of Human Motion in Video](https://mlsanity.com/paper/2207.05845/) (2022)
- [A Gated Attention Transformer for Multi-Person Pose Tracking](https://mlsanity.com/paper/2306.05807/) (2023)
- [Improving Multi-Person Pose Tracking with A Confidence Network](https://mlsanity.com/paper/2310.18920/) (2023)
- [UniHPE: Towards Unified Human Pose Estimation via Contrastive Learning](https://mlsanity.com/paper/2311.16477/) (2023)
- [Waterfall Transformer for Multi-person Pose Estimation](https://mlsanity.com/paper/2411.18944/) (2024)
- [Group Activity Recognition via Dynamic Composition and Interaction](https://mlsanity.com/paper/2305.05583/) (2023)
- [Learning Higher-order Object Interactions for Keypoint-based Video Understanding](https://mlsanity.com/paper/2305.09539/) (2023)

## Abstract (arXiv)

Pose tracking is an important problem that requires identifying unique human pose-instances and matching them temporally across different frames of a video. However, existing pose tracking methods are unable to accurately model temporal relationships and require significant computation, often computing the tracks offline. We present an efficient Multi-person Pose Tracking method, KeyTrack, that only relies on keypoint information without using any RGB or optical flow information to track human keypoints in real-time. Keypoints are tracked using our Pose Entailment method, in which, first, a pair of pose estimates is sampled from different frames in a video and tokenized. Then, a Transformer-based network makes a binary classification as to whether one pose temporally follows another. Furthermore, we improve our top-down pose estimation method with a novel, parameter-free, keypoint refinement technique that improves the keypoint estimates used during the Pose Entailment step. We achieve state-of-the-art results on the PoseTrack'17 and the PoseTrack'18 benchmarks while using only a fraction of the computation required by most other methods for computing the tracking information.