On the role of depth predictions for 3D human pose estimation
Not on the map. Why?
Abstract (arXiv)
Following the successful application of deep convolutional neural networks to 2d human pose estimation, the next logical problem to solve is 3d human pose estimation from monocular images. While previous solutions have shown some success, they do not fully utilize the depth information from the 2d inputs. With the goal of addressing this depth ambiguity, we build a system that takes 2d joint locations as input along with their estimated depth value and predicts their 3d positions in camera coordinates. Given the inherent noise and inaccuracy from estimating depth maps from monocular images, we perform an extensive statistical analysis showing that given this noise there is still a statistically significant correlation between the predicted depth values and the third coordinate of camera coordinates. We further explain how the state-of-the-art results we achieve on the H3.6M validation set are due to the additional input of depth. Notably, our results are produced on neural network that accepts a low dimensional input and be integrated into a real-time system. Furthermore, our system can be combined with an off-the-shelf 2d pose detector and a depth map predictor to perform 3d pose estimation in the wild.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 3 Mar 2021.
Builds on
29 of 45 references.
References · 29
- 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
- 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
- 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
- 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
- 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 494
- 2017A simple yet effective baseline for 3d human pose estimationcited by 401
- 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
- 2016Convolutional Pose Machinescited by 325
- 2017Integral Human Pose Regressioncited by 256
- 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 221
- 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
- 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 132
17 more
- 2017Compositional Human Pose Regressioncited by 124
- 2019Learning the Depths of Moving People by Watching Frozen Peoplecited by 120
- 20163D Human Pose Estimation = 2D Pose Estimation + Matchingcited by 113
- 2018LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Imagescited by 88
- 20183D Human Pose Estimation in the Wild by Adversarial Learningcited by 86
- 20163D Human Pose Estimation from a Single Image via Distance Matrix Regressioncited by 73
- 2017Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Imagecited by 73
- 2016Structured Prediction of 3D Human Pose with Deep Neural Networkscited by 43
- 2016Deep Kinematic Pose Regressioncited by 40
- 2015Maximum-Margin Structured Learning with Deep Networks for 3D Human Pose Estimationcited by 37
- 2017MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Priorcited by 36
- 2017Generating Multiple Diverse Hypotheses for Human 3D Pose Consistent with 2D Joint Detectionscited by 30
- 20163D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Informationcited by 28
- 2015A Dual-Source Approach for 3D Pose Estimation from a Single Imagecited by 9
- 20183D human pose estimation from depth maps using a deep combination of posescited by 6
- 20172D-3D Pose Consistency-based Conditional Random Fields for 3D Human Pose Estimationcited by 3
- 2020Multi-Person Absolute 3D Human Pose Estimation with Weak Depth Supervisioncited by 3
Built on by
Citing papers per year: 2023: 1.
1 paper.