# 3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training

arXiv: https://arxiv.org/abs/2004.11822 · HTML: https://mlsanity.com/paper/2004.11822/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 30 of 54)

- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network](https://mlsanity.com/paper/2203.14564/) (2022)
- [Conditional Directed Graph Convolution for 3D Human Pose Estimation](https://mlsanity.com/paper/2107.07797/) (2021)
- [GLA-GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Video](https://mlsanity.com/paper/2307.05853/) (2023)
- [Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Driving](https://mlsanity.com/paper/2112.12141/) (2021)
- [AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generation](https://mlsanity.com/paper/2112.11593/) (2021)
- [3D-Aware Neural Body Fitting for Occlusion Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2308.10123/) (2023)
- [RobustFusion: Robust Volumetric Performance Reconstruction under Human-object Interactions from Monocular RGBD Stream](https://mlsanity.com/paper/2104.14837/) (2021)
- [Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation](https://mlsanity.com/paper/2203.15293/) (2022)
- [JRDB-Pose: A Large-scale Dataset for Multi-Person Pose Estimation and Tracking](https://mlsanity.com/paper/2210.11940/) (2022)
- [Weakly Supervised 3D Multi-person Pose Estimation for Large-scale Scenes based on Monocular Camera and Single LiDAR](https://mlsanity.com/paper/2211.16951/) (2022)
- [Monocular 3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks](https://mlsanity.com/paper/2104.01797/) (2021)
- [Lifting Monocular Events to 3D Human Poses](https://mlsanity.com/paper/2104.10609/) (2021)
- [AthletePose3D: A Benchmark Dataset for 3D Human Pose Estimation and Kinematic Validation in Athletic Movements](https://mlsanity.com/paper/2503.07499/) (2025)
- [Dual networks based 3D Multi-Person Pose Estimation from Monocular Video](https://mlsanity.com/paper/2205.00748/) (2022)
- [Domain Adaptive 3D Pose Augmentation for In-the-wild Human Mesh Recovery](https://mlsanity.com/paper/2206.10457/) (2022)
- [FLEX: Extrinsic Parameters-free Multi-view 3D Human Motion Reconstruction](https://mlsanity.com/paper/2105.01937/) (2021)
- [Multi-Person 3D Pose and Shape Estimation via Inverse Kinematics and Refinement](https://mlsanity.com/paper/2210.13529/) (2022)
- [Explicit Occlusion Reasoning for Multi-person 3D Human Pose Estimation](https://mlsanity.com/paper/2208.00090/) (2022)
- [3D Pose Estimation and Future Motion Prediction from 2D Images](https://mlsanity.com/paper/2111.13285/) (2021)
- [Spatial-Temporal Parallel Transformer for Arm-Hand Dynamic Estimation](https://mlsanity.com/paper/2203.16202/) (2022)
- [DH-AUG: DH Forward Kinematics Model Driven Augmentation for 3D Human Pose Estimation](https://mlsanity.com/paper/2207.09303/) (2022)
- [Towards Single Camera Human 3D-Kinematics](https://mlsanity.com/paper/2301.05435/) (2023)
- [Probabilistic Triangulation for Uncalibrated Multi-View 3D Human Pose Estimation](https://mlsanity.com/paper/2309.04756/) (2023)
- [IVT: An End-to-End Instance-guided Video Transformer for 3D Pose Estimation](https://mlsanity.com/paper/2208.03431/) (2022)
- [Motion Projection Consistency Based 3D Human Pose Estimation with Virtual Bones from Monocular Videos](https://mlsanity.com/paper/2106.14706/) (2021)
- [Video Based Fall Detection Using Human Poses](https://mlsanity.com/paper/2107.14633/) (2021)
- [Quantification of Occlusion Handling Capability of a 3D Human Pose Estimation Framework](https://mlsanity.com/paper/2203.04113/) (2022)
- [POISE: Pose Guided Human Silhouette Extraction under Occlusions](https://mlsanity.com/paper/2311.05077/) (2023)

## Abstract (arXiv)

Estimating 3D poses from a monocular video is still a challenging task, despite the significant progress that has been made in recent years. Generally, the performance of existing methods drops when the target person is too small/large, or the motion is too fast/slow relative to the scale and speed of the training data. Moreover, to our knowledge, many of these methods are not designed or trained under severe occlusion explicitly, making their performance on handling occlusion compromised. Addressing these problems, we introduce a spatio-temporal network for robust 3D human pose estimation. As humans in videos may appear in different scales and have various motion speeds, we apply multi-scale spatial features for 2D joints or keypoints prediction in each individual frame, and multi-stride temporal convolutional net-works (TCNs) to estimate 3D joints or keypoints. Furthermore, we design a spatio-temporal discriminator based on body structures as well as limb motions to assess whether the predicted pose forms a valid pose and a valid movement. During training, we explicitly mask out some keypoints to simulate various occlusion cases, from minor to severe occlusion, so that our network can learn better and becomes robust to various degrees of occlusion. As there are limited 3D ground-truth data, we further utilize 2D video data to inject a semi-supervised learning capability to our network. Experiments on public datasets validate the effectiveness of our method, and our ablation studies show the strengths of our network's individual submodules.