# SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos

arXiv: https://arxiv.org/abs/2501.15073 · HTML: https://mlsanity.com/paper/2501.15073/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 43 references that are in the map.
- First posted on arXiv 25 Jan 2025.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [InternVideo: General Video Foundation Models via Generative and Discriminative Learning](https://mlsanity.com/paper/2212.03191/) (2022)
- [Activating More Pixels in Image Super-Resolution Transformer](https://mlsanity.com/paper/2205.04437/) (2022)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Pose Flow: Efficient Online Pose Tracking](https://mlsanity.com/paper/1802.00977/) (2018)
- [Flowing ConvNets for Human Pose Estimation in Videos](https://mlsanity.com/paper/1506.02897/) (2015)
- [Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolution](https://mlsanity.com/paper/2204.13062/) (2022)
- [Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification](https://mlsanity.com/paper/2104.02862/) (2021)
- [UniPose: Unified Human Pose Estimation in Single Images and Videos](https://mlsanity.com/paper/2001.08095/) (2020)
- [Combining detection and tracking for human pose estimation in videos](https://mlsanity.com/paper/2003.13743/) (2020)
- [PoseTrack: Joint Multi-Person Pose Estimation and Tracking](https://mlsanity.com/paper/1611.07727/) (2016)
- [Learning Temporal Pose Estimation from Sparsely-Labeled Videos](https://mlsanity.com/paper/1906.04016/) (2019)
- [Unsupervised Human Pose Estimation through Transforming Shape Templates](https://mlsanity.com/paper/2105.04154/) (2021)
- [Locate and Verify: A Two-Stream Network for Improved Deepfake Detection](https://mlsanity.com/paper/2309.11131/) (2023)
- [Mutual Information-Based Temporal Difference Learning for Human Pose Estimation in Video](https://mlsanity.com/paper/2303.08475/) (2023)
- [Motion Prediction via Joint Dependency Modeling in Phase Space](https://mlsanity.com/paper/2201.02365/) (2022)
- [Copy Motion From One to Another: Fake Motion Video Generation](https://mlsanity.com/paper/2205.01373/) (2022)
- [OTPose: Occlusion-Aware Transformer for Pose Estimation in Sparsely-Labeled Videos](https://mlsanity.com/paper/2207.09725/) (2022)
- [Video-Based Human Pose Regression via Decoupled Space-Time Aggregation](https://mlsanity.com/paper/2403.19926/) (2024)
- [Do As I Do: Pose Guided Human Motion Copy](https://mlsanity.com/paper/2406.16601/) (2024)

## Built on by (most cited first, 3 of 3)

- [KineST: A Kinematics-guided Spatiotemporal State Space Model for Human Motion Tracking from Sparse Signals](https://mlsanity.com/paper/2512.16791/) (2025)
- [Multi-Grained Feature Pruning for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2503.05365/) (2025)
- [NCSTR: Node-Centric Decoupled Spatio-Temporal Reasoning for Video-based Human Pose Estimation](https://mlsanity.com/paper/2603.20323/) (2026)

## Abstract (arXiv)

Human pose estimation in videos remains a challenge, largely due to the reliance on extensive manual annotation of large datasets, which is expensive and labor-intensive. Furthermore, existing approaches often struggle to capture long-range temporal dependencies and overlook the complementary relationship between temporal pose heatmaps and visual features. To address these limitations, we introduce STDPose, a novel framework that enhances human pose estimation by learning spatiotemporal dynamics in sparsely-labeled videos. STDPose incorporates two key innovations: 1) A novel Dynamic-Aware Mask to capture long-range motion context, allowing for a nuanced understanding of pose changes. 2) A system for encoding and aggregating spatiotemporal representations and motion dynamics to effectively model spatiotemporal relationships, improving the accuracy and robustness of pose estimation. STDPose establishes a new performance benchmark for both video pose propagation (i.e., propagating pose annotations from labeled frames to unlabeled frames) and pose estimation tasks, across three large-scale evaluation datasets. Additionally, utilizing pseudo-labels generated by pose propagation, STDPose achieves competitive performance with only 26.7% labeled data.