# OTPose: Occlusion-Aware Transformer for Pose Estimation in Sparsely-Labeled Videos

arXiv: https://arxiv.org/abs/2207.09725 · HTML: https://mlsanity.com/paper/2207.09725/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 20 Jul 2022.

## Builds on (most cited first, 19 of 19)

- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [PoseTrack: A Benchmark for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1710.10000/) (2017)
- [TransPose: Keypoint Localization via Transformer](https://mlsanity.com/paper/2012.14214/) (2020)
- [Pose Flow: Efficient Online Pose Tracking](https://mlsanity.com/paper/1802.00977/) (2018)
- [Deep Dual Consecutive Network for Human Pose Estimation](https://mlsanity.com/paper/2103.07254/) (2021)
- [Combining detection and tracking for human pose estimation in videos](https://mlsanity.com/paper/2003.13743/) (2020)
- [PoseTrack: Joint Multi-Person Pose Estimation and Tracking](https://mlsanity.com/paper/1611.07727/) (2016)
- [Learning Temporal Pose Estimation from Sparsely-Labeled Videos](https://mlsanity.com/paper/1906.04016/) (2019)
- [LSTM Pose Machines](https://mlsanity.com/paper/1712.06316/) (2017)
- [Multi-Domain Pose Network for Multi-Person Pose Estimation and Tracking](https://mlsanity.com/paper/1810.08338/) (2018)
- [Joint Flow: Temporal Flow Fields for Multi Person Tracking](https://mlsanity.com/paper/1805.04596/) (2018)

## Built on by (most cited first, 10 of 10)

- [Video-Based Human Pose Regression via Decoupled Space-Time Aggregation](https://mlsanity.com/paper/2403.19926/) (2024)
- [VisionKG: Unleashing the Power of Visual Datasets via Knowledge Graph](https://mlsanity.com/paper/2309.13610/) (2023)
- [Kinematic-aware Hierarchical Attention Network for Human Pose Estimation in Videos](https://mlsanity.com/paper/2211.15868/) (2022)
- [Active Transfer Learning for Efficient Video-Specific Human Pose Estimation](https://mlsanity.com/paper/2311.05041/) (2023)
- [SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos](https://mlsanity.com/paper/2501.15073/) (2025)
- [Enhancing Video Large Language Models with Structured Multi-Video Collaborative Reasoning](https://mlsanity.com/paper/2509.13161/) (2025)
- [MamKPD: A Simple Mamba Baseline for Real-Time 2D Keypoint Detection](https://mlsanity.com/paper/2412.01422/) (2024)
- [End-to-End Multi-Person Pose Estimation with Pose-Aware Video Transformer](https://mlsanity.com/paper/2511.13208/) (2025)
- [Multi-Grained Feature Pruning for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2503.05365/) (2025)
- [Beyond Static Frames: Temporal Aggregate-and-Restore Vision Transformer for Human Pose Estimation](https://mlsanity.com/paper/2603.05929/) (2026)

## Abstract (arXiv)

Although many approaches for multi-human pose estimation in videos have shown profound results, they require densely annotated data which entails excessive man labor. Furthermore, there exists occlusion and motion blur that inevitably lead to poor estimation performance. To address these problems, we propose a method that leverages an attention mask for occluded joints and encodes temporal dependency between frames using transformers. First, our framework composes different combinations of sparsely annotated frames that denote the track of the overall joint movement. We propose an occlusion attention mask from these combinations that enable encoding occlusion-aware heatmaps as a semi-supervised task. Second, the proposed temporal encoder employs transformer architecture to effectively aggregate the temporal relationship and keypoint-wise attention from each time step and accurately refines the target frame's final pose estimation. We achieve state-of-the-art pose estimation results for PoseTrack2017 and PoseTrack2018 datasets and demonstrate the robustness of our approach to occlusion and motion blur in sparsely annotated video data.