# Learning Temporal Pose Estimation from Sparsely-Labeled Videos

arXiv: https://arxiv.org/abs/1906.04016 · HTML: https://mlsanity.com/paper/1906.04016/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2019.

## Built on by (most cited first, 30 of 39)

- [AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time](https://mlsanity.com/paper/2211.03375/) (2022)
- [Track to Detect and Segment: An Online Multi-Object Tracker](https://mlsanity.com/paper/2103.08808/) (2021)
- [TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments](https://mlsanity.com/paper/2306.02850/) (2023)
- [Deep Dual Consecutive Network for Human Pose Estimation](https://mlsanity.com/paper/2103.07254/) (2021)
- [Learning Dynamics via Graph Neural Networks for Human Pose Estimation and Tracking](https://mlsanity.com/paper/2106.03772/) (2021)
- [DiffPose: SpatioTemporal Diffusion Model for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2307.16687/) (2023)
- [Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2203.15227/) (2022)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [Vision-Based Human Pose Estimation via Deep Learning: A Survey](https://mlsanity.com/paper/2308.13872/) (2023)
- [Mutual Information-Based Temporal Difference Learning for Human Pose Estimation in Video](https://mlsanity.com/paper/2303.08475/) (2023)
- [OTPose: Occlusion-Aware Transformer for Pose Estimation in Sparsely-Labeled Videos](https://mlsanity.com/paper/2207.09725/) (2022)
- [Video-Based Human Pose Regression via Decoupled Space-Time Aggregation](https://mlsanity.com/paper/2403.19926/) (2024)
- [H-MoRe: Learning Human-centric Motion Representation for Action Analysis](https://mlsanity.com/paper/2504.10676/) (2025)
- [Joint-Motion Mutual Learning for Pose Estimation in Videos](https://mlsanity.com/paper/2408.02285/) (2024)
- [Kinematic-aware Hierarchical Attention Network for Human Pose Estimation in Videos](https://mlsanity.com/paper/2211.15868/) (2022)
- [Active Transfer Learning for Efficient Video-Specific Human Pose Estimation](https://mlsanity.com/paper/2311.05041/) (2023)
- [High-Resolution Spatiotemporal Modeling with Global-Local State Space Models for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2510.11017/) (2025)
- [Cell Detection from Imperfect Annotation by Pseudo Label Selection Using P-classification](https://mlsanity.com/paper/2107.09289/) (2021)
- [Improving Multi-Person Pose Tracking with A Confidence Network](https://mlsanity.com/paper/2310.18920/) (2023)
- [SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos](https://mlsanity.com/paper/2501.15073/) (2025)
- [Temporally Guided Articulated Hand Pose Tracking in Surgical Videos](https://mlsanity.com/paper/2101.04281/) (2021)
- [HoughNet: Integrating near and long-range evidence for visual detection](https://mlsanity.com/paper/2104.06773/) (2021)
- [SemiMultiPose: A Semi-supervised Multi-animal Pose Estimation Framework](https://mlsanity.com/paper/2204.07072/) (2022)
- [Joint Coordinate Regression and Association For Multi-Person Pose Estimation, A Pure Neural Network Approach](https://mlsanity.com/paper/2307.01004/) (2023)
- [Semi-supervised Cell Detection in Time-lapse Images Using Temporal Consistency](https://mlsanity.com/paper/2107.08639/) (2021)
- [Semi-Supervised 3D Hand Shape and Pose Estimation with Label Propagation](https://mlsanity.com/paper/2111.15199/) (2021)
- [SigFormer: Sparse Signal-Guided Transformer for Multi-Modal Human Action Segmentation](https://mlsanity.com/paper/2311.17428/) (2023)
- [End-to-End Multi-Person Pose Estimation with Pose-Aware Video Transformer](https://mlsanity.com/paper/2511.13208/) (2025)
- [ShadowDraw: From Any Object to Shadow-Drawing Compositional Art](https://mlsanity.com/paper/2512.05110/) (2025)
- [SportSkills: Physical Skill Learning from Sports Instructional Videos](https://mlsanity.com/paper/2603.25163/) (2026)

## Abstract (arXiv)

Modern approaches for multi-person pose estimation in video require large amounts of dense annotations. However, labeling every frame in a video is costly and labor intensive. To reduce the need for dense annotations, we propose a PoseWarper network that leverages training videos with sparse annotations (every k frames) to learn to perform dense temporal pose propagation and estimation. Given a pair of video frames---a labeled Frame A and an unlabeled Frame B---we train our model to predict human pose in Frame A using the features from Frame B by means of deformable convolutions to implicitly learn the pose warping between A and B. We demonstrate that we can leverage our trained PoseWarper for several applications. First, at inference time we can reverse the application direction of our network in order to propagate pose information from manually annotated frames to unlabeled frames. This makes it possible to generate pose annotations for the entire video given only a few manually-labeled frames. Compared to modern label propagation methods based on optical flow, our warping mechanism is much more compact (6M vs 39M parameters), and also more accurate (88.7% mAP vs 83.8% mAP). We also show that we can improve the accuracy of a pose estimator by training it on an augmented dataset obtained by adding our propagated poses to the original manual labels. Lastly, we can use our PoseWarper to aggregate temporal pose information from neighboring frames during inference. This allows our system to achieve state-of-the-art pose detection results on the PoseTrack2017 and PoseTrack2018 datasets. Code has been made available at: https://github.com/facebookresearch/PoseWarper.