# Self-supervised Keypoint Correspondences for Multi-Person Pose Estimation and Tracking in Videos

arXiv: https://arxiv.org/abs/2004.12652 · HTML: https://mlsanity.com/paper/2004.12652/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2020.

## Built on by (most cited first, 22 of 22)

- [ViTPose++: Vision Transformer for Generic Body Pose Estimation](https://mlsanity.com/paper/2212.04246/) (2022)
- [SiLK -- Simple Learned Keypoints](https://mlsanity.com/paper/2304.06194/) (2023)
- [Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective](https://mlsanity.com/paper/2104.11536/) (2021)
- [DiffPose: SpatioTemporal Diffusion Model for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2307.16687/) (2023)
- [Unified Keypoint-based Action Recognition Framework via Structured Keypoint Pooling](https://mlsanity.com/paper/2303.15270/) (2023)
- [Mutual Information-Based Temporal Difference Learning for Human Pose Estimation in Video](https://mlsanity.com/paper/2303.08475/) (2023)
- [Monocular 3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks](https://mlsanity.com/paper/2104.01797/) (2021)
- [Dual networks based 3D Multi-Person Pose Estimation from Monocular Video](https://mlsanity.com/paper/2205.00748/) (2022)
- [Keypoint Message Passing for Video-based Person Re-Identification](https://mlsanity.com/paper/2111.08279/) (2021)
- [Joint-Motion Mutual Learning for Pose Estimation in Videos](https://mlsanity.com/paper/2408.02285/) (2024)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [High-Resolution Spatiotemporal Modeling with Global-Local State Space Models for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2510.11017/) (2025)
- [A Gated Attention Transformer for Multi-Person Pose Tracking](https://mlsanity.com/paper/2306.05807/) (2023)
- [Improving Multi-Person Pose Tracking with A Confidence Network](https://mlsanity.com/paper/2310.18920/) (2023)
- [Event-based Solutions for Human-centered Applications: A Comprehensive Review](https://mlsanity.com/paper/2502.18490/) (2025)
- [Do as we do: Multiple Person Video-To-Video Transfer](https://mlsanity.com/paper/2104.04721/) (2021)
- [DINTR: Tracking via Diffusion-based Interpolation](https://mlsanity.com/paper/2410.10053/) (2024)
- [WAIT: Feature Warping for Animation to Illustration video Translation using GANs](https://mlsanity.com/paper/2310.04901/) (2023)
- [Neural Texture Puppeteer: A Framework for Neural Geometry and Texture Rendering of Articulated Shapes, Enabling Re-Identification at Interactive Speed](https://mlsanity.com/paper/2311.17109/) (2023)
- [Matching Semantically Similar Non-Identical Objects](https://mlsanity.com/paper/2403.08227/) (2024)
- [Learning semantical dynamics and spatiotemporal collaboration for human pose estimation in video](https://mlsanity.com/paper/2502.10616/) (2025)
- [An End-to-End Framework for Video Multi-Person Pose Estimation](https://mlsanity.com/paper/2509.01095/) (2025)

## Abstract (arXiv)

Video annotation is expensive and time consuming. Consequently, datasets for multi-person pose estimation and tracking are less diverse and have more sparse annotations compared to large scale image datasets for human pose estimation. This makes it challenging to learn deep learning based models for associating keypoints across frames that are robust to nuisance factors such as motion blur and occlusions for the task of multi-person pose tracking. To address this issue, we propose an approach that relies on keypoint correspondences for associating persons in videos. Instead of training the network for estimating keypoint correspondences on video data, it is trained on a large scale image datasets for human pose estimation using self-supervision. Combined with a top-down framework for human pose estimation, we use keypoints correspondences to (i) recover missed pose detections (ii) associate pose detections across video frames. Our approach achieves state-of-the-art results for multi-frame pose estimation and multi-person pose tracking on the PosTrack 2017 and PoseTrack 2018 data sets.